[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:21 dist-test-slave-tm1g krb5kdc[17634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:21 dist-test-slave-tm1g krb5kdc[17634](info): set up 2 sockets
Aug 12 03:52:21 dist-test-slave-tm1g krb5kdc[17634](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 03:52:22.634138 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:22 dist-test-slave-tm1g krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:22.763357 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33555
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:33555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:22.852077 17650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:22.852351 17650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:22.852396 17650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:22.853611 17650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:22.853688 17650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:22.853713 17650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:22.853732 17650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:22.853760 17650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:22.855580 17650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:33555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:22.856047 17650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:22.856345 17650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:22.864677 17659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:22.864711 17650 server_base.cc:1034] running on GCE node
W20260812 03:52:22.864953 17656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:22.864995 17657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:22.865340 17650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:22.865841 17650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:22.867007 17650 hybrid_clock.cc:648] HybridClock initialized: now 1786506742867000 us; error 55 us; skew 500 ppm
Aug 12 03:52:22 dist-test-slave-tm1g krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506742, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:22.869727 17650 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:22.870489 17650 webserver.cc:460] Webserver started at http://127.17.54.254:33381/ using document root <none> and password file <none>
I20260812 03:52:22.870795 17650 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:22.870846 17650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:22.871084 17650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:22.872325 17650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "2a8cf0008f0d4831ae57516f358ef022"
format_stamp: "Formatted at 2026-08-12 03:52:22 on dist-test-slave-tm1g"
server_key: "3aaca3ab9d110349380f47de97a33634"
server_key_iv: "1f028818f6812a13d0269c61337b9f8b"
server_key_version: "encryptionkey@0"
I20260812 03:52:22.872701 17650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "2a8cf0008f0d4831ae57516f358ef022"
format_stamp: "Formatted at 2026-08-12 03:52:22 on dist-test-slave-tm1g"
server_key: "3aaca3ab9d110349380f47de97a33634"
server_key_iv: "1f028818f6812a13d0269c61337b9f8b"
server_key_version: "encryptionkey@0"
I20260812 03:52:22.874845 17650 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 03:52:22.876103 17669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:22.876435 17650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:22.876565 17650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "2a8cf0008f0d4831ae57516f358ef022"
format_stamp: "Formatted at 2026-08-12 03:52:22 on dist-test-slave-tm1g"
server_key: "3aaca3ab9d110349380f47de97a33634"
server_key_iv: "1f028818f6812a13d0269c61337b9f8b"
server_key_version: "encryptionkey@0"
I20260812 03:52:22.876665 17650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:22.896857 17650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:22.899542 17650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:22.899765 17650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:22.905097 17650 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:33555
I20260812 03:52:22.905243 17725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:33555 every 8 connection(s)
I20260812 03:52:22.905833 17650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:22.907389 17727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:22.909545 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 17650
I20260812 03:52:22.909679 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:22.910420 17627 external_mini_cluster.cc:1428] Setting key 10868981b73b296312256df4bd891c1e
I20260812 03:52:22.910904 17727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022: Bootstrap starting.
I20260812 03:52:22.912161 17727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:22.912626 17727 log.cc:824] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:22.913465 17727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022: No bootstrap required, opened a new log
I20260812 03:52:22.915589 17727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8cf0008f0d4831ae57516f358ef022" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33555 } }
I20260812 03:52:22.915825 17727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:22.916005 17727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8cf0008f0d4831ae57516f358ef022, State: Initialized, Role: FOLLOWER
I20260812 03:52:22.916222 17727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8cf0008f0d4831ae57516f358ef022" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33555 } }
I20260812 03:52:22.916378 17727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:22.916404 17727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:22.916472 17727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:22.917568 17727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8cf0008f0d4831ae57516f358ef022" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33555 } }
I20260812 03:52:22.917790 17727 leader_election.cc:302] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a8cf0008f0d4831ae57516f358ef022; no voters:
I20260812 03:52:22.918027 17727 leader_election.cc:288] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:22.918088 17733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:22.918262 17733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [term 1 LEADER]: Becoming Leader. State: Replica: 2a8cf0008f0d4831ae57516f358ef022, State: Running, Role: LEADER
I20260812 03:52:22.918435 17733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8cf0008f0d4831ae57516f358ef022" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33555 } }
I20260812 03:52:22.918679 17727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:22.918959 17733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a8cf0008f0d4831ae57516f358ef022. Latest consensus state: current_term: 1 leader_uuid: "2a8cf0008f0d4831ae57516f358ef022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8cf0008f0d4831ae57516f358ef022" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33555 } } }
I20260812 03:52:22.918967 17735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a8cf0008f0d4831ae57516f358ef022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8cf0008f0d4831ae57516f358ef022" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33555 } } }
I20260812 03:52:22.919137 17733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:22.919149 17735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:22.919504 17741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:22.920719 17741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:22.923338 17741 catalog_manager.cc:1347] Generated new cluster ID: a405c37c043f4a6c89a56e494dd0a0e1
I20260812 03:52:22.923425 17741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:52:22 dist-test-slave-tm1g krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:22.931468 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:22.911877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46466 (local address 127.17.54.254:33555)
0812 03:52:22.912258 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:22.912269 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:22.912472 (+ 203us) server_negotiation.cc:407] Connection header received
0812 03:52:22.915501 (+ 3029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:22.915509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:22.915627 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:22.915813 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:22.917592 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:22.919042 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:22.921008 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:22.921844 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:22.921981 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:22.926943 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:22.926947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:22.926948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:22.926961 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:22.929086 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:22.929597 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:22.929601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:22.929601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:22.929747 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:22.930038 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:22.930039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:22.930039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:22.930411 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:22.930596 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:22.930894 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 03:52:22.931142 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":107,"threads_started":1}
I20260812 03:52:22.940088 17741 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:22.940719 17741 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:22.945734 17741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a8cf0008f0d4831ae57516f358ef022: Generated new TSK 0
I20260812 03:52:22.945958 17741 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:22.982261 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33555
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:23.101922 17761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:23.102190 17761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:23.102236 17761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:23.103782 17761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:23.103957 17761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:23.104030 17761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:23.105996 17761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33555
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:23.106470 17761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:23.106770 17761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:23.114471 17768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:23.114516 17767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:23.114471 17770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:23.114886 17761 server_base.cc:1034] running on GCE node
I20260812 03:52:23.115108 17761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:23.115425 17761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:23.116634 17761 hybrid_clock.cc:648] HybridClock initialized: now 1786506743116607 us; error 43 us; skew 500 ppm
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:23.121359 17761 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:23.121924 17761 webserver.cc:460] Webserver started at http://127.17.54.193:41567/ using document root <none> and password file <none>
I20260812 03:52:23.122190 17761 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:23.122239 17761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:23.122401 17761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:23.123571 17761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "182c89e3c9af40328a84a0c544803c0f"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "63882a67fed73bf777bb6314c5100da8"
server_key_iv: "34992d1a9b676eae25ee65fd188ee545"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.123963 17761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "182c89e3c9af40328a84a0c544803c0f"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "63882a67fed73bf777bb6314c5100da8"
server_key_iv: "34992d1a9b676eae25ee65fd188ee545"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.125520 17761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:23.126529 17780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.126734 17761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:23.126847 17761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "182c89e3c9af40328a84a0c544803c0f"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "63882a67fed73bf777bb6314c5100da8"
server_key_iv: "34992d1a9b676eae25ee65fd188ee545"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.126942 17761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:23.137442 17761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:23.140136 17761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:23.140311 17761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:23.140661 17761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:23.141088 17761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:23.141192 17761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.141242 17761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:23.141273 17761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.153920 17761 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:34925
I20260812 03:52:23.154035 17909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:34925 every 8 connection(s)
I20260812 03:52:23.154479 17761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:23.160792 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 17761
I20260812 03:52:23.160913 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:23.161085 17627 external_mini_cluster.cc:1428] Setting key 49a2004dd4fd11dd5d91493eef3a2782
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:23.167168 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:23.155599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:33543 (local address 127.17.54.254:33555)
0812 03:52:23.155655 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:23.155657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:23.156005 (+ 348us) server_negotiation.cc:407] Connection header received
0812 03:52:23.156709 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:23.156710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:23.156728 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:23.156804 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:23.158516 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.159038 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.160283 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.160512 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.160612 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:23.163940 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:23.163942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:23.163943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:23.163952 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:23.165532 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.166011 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.166013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.166013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.166080 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.166407 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.166408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.166409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.166538 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:23.166616 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:23.166939 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 03:52:23.167066 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:52:23.167337 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:23.155501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33555 (local address 127.17.54.193:33543)
0812 03:52:23.155751 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:23.155798 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:23.156534 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:23.157109 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:23.157115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:23.157991 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:23.158294 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:23.158347 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.159160 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.159161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:23.160157 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:23.160160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.160753 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.160755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:23.160974 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:23.161757 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:23.161776 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:23.163830 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:23.165642 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:23.165652 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:23.165655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:23.165910 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:23.166167 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:23.166169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:23.166169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:23.166292 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:23.166620 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:23.166629 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:23.166784 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 03:52:23.166894 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 03:52:23.167076 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
I20260812 03:52:23.168064 17910 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33555
I20260812 03:52:23.168184 17910 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:23.168381 17910 heartbeater.cc:511] Master 127.17.54.254:33555 requested a full tablet report, sending...
I20260812 03:52:23.169497 17687 ts_manager.cc:194] Registered new tserver with Master: 182c89e3c9af40328a84a0c544803c0f (127.17.54.193:34925)
I20260812 03:52:23.170787 17687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:33543
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:23.194602 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33555
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:23.317999 17915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:23.318259 17915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:23.318305 17915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:23.319960 17915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:23.320011 17915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:23.320070 17915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:23.321890 17915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33555
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:23.322108 17915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:23.322449 17915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:23.328032 17920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:23.328066 17922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:23.328248 17924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:23.328550 17915 server_base.cc:1034] running on GCE node
I20260812 03:52:23.328804 17915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:23.329211 17915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:23.330456 17915 hybrid_clock.cc:648] HybridClock initialized: now 1786506743330428 us; error 52 us; skew 500 ppm
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:23.334000 17915 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:23.334579 17915 webserver.cc:460] Webserver started at http://127.17.54.194:46609/ using document root <none> and password file <none>
I20260812 03:52:23.334851 17915 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:23.334900 17915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:23.335045 17915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:23.336328 17915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "2f0410a24ab8425c93e10c4ca273fda6"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "6bbd21eae4b1cc99c31a8f7acd44f570"
server_key_iv: "6eae6953d6ee4750bc4a3c123af3baa5"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.336776 17915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "2f0410a24ab8425c93e10c4ca273fda6"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "6bbd21eae4b1cc99c31a8f7acd44f570"
server_key_iv: "6eae6953d6ee4750bc4a3c123af3baa5"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.338361 17915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 03:52:23.339443 17933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.339687 17915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:23.339807 17915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "2f0410a24ab8425c93e10c4ca273fda6"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "6bbd21eae4b1cc99c31a8f7acd44f570"
server_key_iv: "6eae6953d6ee4750bc4a3c123af3baa5"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.339924 17915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:23.348503 17915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:23.351188 17915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:23.351457 17915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:23.351965 17915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:23.352370 17915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:23.352437 17915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.352496 17915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:23.352526 17915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.365535 17915 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:35635
I20260812 03:52:23.365715 18056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:35635 every 8 connection(s)
I20260812 03:52:23.366025 17915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:52:23.371454 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 17915
I20260812 03:52:23.371558 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:23.371759 17627 external_mini_cluster.cc:1428] Setting key 41970bc0ce9be6b3e930a550e76edf5a
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:23.377499 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:23.367064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:54961 (local address 127.17.54.254:33555)
0812 03:52:23.367114 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:23.367117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:23.367540 (+ 423us) server_negotiation.cc:407] Connection header received
0812 03:52:23.368076 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:23.368078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:23.368090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:23.368147 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:23.369181 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.369704 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.370954 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.371170 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.371257 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:23.374154 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:23.374156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:23.374156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:23.374165 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:23.376054 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.376470 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.376471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.376472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.376541 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.376815 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.376816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.376817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.376945 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:23.377026 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:23.377308 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 03:52:23.377399 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:52:23.377719 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:23.367004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33555 (local address 127.17.54.194:54961)
0812 03:52:23.367361 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:23.367429 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:23.367988 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:23.368202 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:23.368205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:23.368833 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:23.369083 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:23.369088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.369808 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.369809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:23.370846 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:23.370850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.371307 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.371308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:23.371547 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:23.372246 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:23.372255 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:23.374044 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:23.376138 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:23.376146 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:23.376148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:23.376382 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:23.376619 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:23.376620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:23.376620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:23.376734 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:23.377014 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:23.377022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:23.377172 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 03:52:23.377286 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:52:23.377453 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":233,"threads_started":1}
I20260812 03:52:23.378284 18057 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33555
I20260812 03:52:23.378429 18057 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:23.378669 18057 heartbeater.cc:511] Master 127.17.54.254:33555 requested a full tablet report, sending...
I20260812 03:52:23.379138 17687 ts_manager.cc:194] Registered new tserver with Master: 2f0410a24ab8425c93e10c4ca273fda6 (127.17.54.194:35635)
I20260812 03:52:23.379529 17687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:54961
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:23.402482 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33555
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:23.595914 18062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:23.596272 18062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:23.596319 18062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:23.598158 18062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:23.598243 18062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:23.598290 18062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:23.601102 18062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33555
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:23.601426 18062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:23.601872 18062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:23.610924 18074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:23.610965 18071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:23.610988 18070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:23.611528 18062 server_base.cc:1034] running on GCE node
I20260812 03:52:23.611845 18062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:23.612298 18062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:23.613571 18062 hybrid_clock.cc:648] HybridClock initialized: now 1786506743613539 us; error 49 us; skew 500 ppm
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:23.623015 18062 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:23.623711 18062 webserver.cc:460] Webserver started at http://127.17.54.195:40833/ using document root <none> and password file <none>
I20260812 03:52:23.624032 18062 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:23.624120 18062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:23.624298 18062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:23.626143 18062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "69ffe1776e8641e294e1ad66d9ed8732"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "4dcde71d7acc6574bb5b08894dc8d5b1"
server_key_iv: "346e4cf4c7bfd15f273f64db0a362c83"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.626946 18062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "69ffe1776e8641e294e1ad66d9ed8732"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "4dcde71d7acc6574bb5b08894dc8d5b1"
server_key_iv: "346e4cf4c7bfd15f273f64db0a362c83"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.629817 18062 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:52:23.631505 18081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.631865 18062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:23.632020 18062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "69ffe1776e8641e294e1ad66d9ed8732"
format_stamp: "Formatted at 2026-08-12 03:52:23 on dist-test-slave-tm1g"
server_key: "4dcde71d7acc6574bb5b08894dc8d5b1"
server_key_iv: "346e4cf4c7bfd15f273f64db0a362c83"
server_key_version: "encryptionkey@0"
I20260812 03:52:23.632157 18062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:23.655120 18062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:23.659626 18062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:23.659972 18062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:23.660678 18062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:23.661309 18062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:23.661399 18062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.661468 18062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:23.661553 18062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:23.686271 18062 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:42131
I20260812 03:52:23.686591 18206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:42131 every 8 connection(s)
I20260812 03:52:23.687049 18062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:23.693969 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 18062
I20260812 03:52:23.694093 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:23.694517 17627 external_mini_cluster.cc:1428] Setting key 67e7cd3750e64f5e917122a367e2ff9b
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:23.704391 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:23.688572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:56319 (local address 127.17.54.254:33555)
0812 03:52:23.688677 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:23.688679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:23.689003 (+ 324us) server_negotiation.cc:407] Connection header received
0812 03:52:23.689908 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:23.689910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:23.689935 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:23.690004 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:23.691872 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.692622 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.694580 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.694961 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.695091 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:23.699271 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:23.699272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:23.699273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:23.699281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:23.702010 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.702957 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.702959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.702959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.703040 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.703362 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.703362 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.703363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.703633 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:23.703784 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:23.704181 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 03:52:23.704300 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:52:23.704927 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:23.688533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33555 (local address 127.17.54.195:56319)
0812 03:52:23.688801 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:23.688865 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:23.689710 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:23.690145 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:23.690149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:23.691247 (+ 1098us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:23.691653 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:23.691658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.692822 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.692823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:23.694424 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:23.694427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.695192 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.695194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:23.695522 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:23.696534 (+ 1012us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:23.696543 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:23.699155 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:23.702274 (+ 3119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:23.702286 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:23.702289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:23.702814 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:23.703117 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:23.703117 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:23.703118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:23.703280 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:23.703754 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:23.703758 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:23.703967 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 03:52:23.704136 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 03:52:23.704352 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":169,"threads_started":1}
I20260812 03:52:23.705839 18207 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33555
I20260812 03:52:23.706064 18207 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:23.706506 18207 heartbeater.cc:511] Master 127.17.54.254:33555 requested a full tablet report, sending...
I20260812 03:52:23.707216 17687 ts_manager.cc:194] Registered new tserver with Master: 69ffe1776e8641e294e1ad66d9ed8732 (127.17.54.195:42131)
I20260812 03:52:23.707901 17687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:56319
I20260812 03:52:23.708437 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:23.737551 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:23.728344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.17.54.193:34925)
0812 03:52:23.728521 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:23.728528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:23.728615 (+ 87us) server_negotiation.cc:407] Connection header received
0812 03:52:23.728718 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:23.728723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:23.728743 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:23.728849 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:23.729639 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.730609 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.731378 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.731769 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.731862 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:23.734376 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:23.734386 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:23.734387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:23.734394 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:23.736104 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.736505 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.736509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.736509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.736579 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.736789 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.736790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.736790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.737007 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:23.737167 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:23.737254 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 03:52:23.737342 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
W20260812 03:52:23.738170 17811 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60896
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:23.766685 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:23.759312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60906 (local address 127.17.54.193:34925)
0812 03:52:23.759385 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:23.759388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:23.759487 (+ 99us) server_negotiation.cc:407] Connection header received
0812 03:52:23.759596 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:23.759597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:23.759613 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:23.759672 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:23.760284 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.760795 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.761607 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:23.761824 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:23.761912 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:23.764309 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:23.764310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:23.764310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:23.764319 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:23.765519 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.765863 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.765864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.765864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.765947 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:23.766148 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:23.766149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:23.766150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:23.766257 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:23.766388 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:23.766487 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 03:52:23.766573 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:52:23.767268 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 17761
I20260812 03:52:23.774757 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 17915
I20260812 03:52:23.784149 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 18062
I20260812 03:52:23.790400 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 17650
2026-08-12T03:52:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2311 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[18225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[18225](info): set up 2 sockets
Aug 12 03:52:23 dist-test-slave-tm1g krb5kdc[18225](info): commencing operation
krb5kdc: starting...
W20260812 03:52:24.912048 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:24 dist-test-slave-tm1g krb5kdc[18225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:25.046813 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:43493
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:43493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:25.132787 18240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:25.133062 18240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:25.133100 18240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:25.134364 18240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:25.134439 18240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:25.134462 18240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:25.134485 18240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:25.134516 18240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:25.136047 18240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:43493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:43493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:25.136339 18240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:25.136590 18240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:25.141919 18246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:25.142158 18248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:25.142287 18245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:25.142272 18240 server_base.cc:1034] running on GCE node
I20260812 03:52:25.142611 18240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:25.142998 18240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:25.144176 18240 hybrid_clock.cc:648] HybridClock initialized: now 1786506745144162 us; error 45 us; skew 500 ppm
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:25.147204 18240 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:25.147711 18240 webserver.cc:460] Webserver started at http://127.17.54.254:46023/ using document root <none> and password file <none>
I20260812 03:52:25.147984 18240 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:25.148062 18240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:25.148216 18240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:25.149326 18240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "00db0b68eef444b6a2786fac18a61775"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "1b7855f6a662220db67bacc8cc3a2420"
server_key_iv: "41d4ff3ff228fe47ce1eaf068cf13ca6"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.149698 18240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "00db0b68eef444b6a2786fac18a61775"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "1b7855f6a662220db67bacc8cc3a2420"
server_key_iv: "41d4ff3ff228fe47ce1eaf068cf13ca6"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.151247 18240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:25.152264 18258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.152544 18240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:25.152653 18240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "00db0b68eef444b6a2786fac18a61775"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "1b7855f6a662220db67bacc8cc3a2420"
server_key_iv: "41d4ff3ff228fe47ce1eaf068cf13ca6"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.152743 18240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:25.181787 18240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:25.184399 18240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:25.184607 18240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:25.188838 18240 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:43493
I20260812 03:52:25.188875 18315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:43493 every 8 connection(s)
I20260812 03:52:25.189262 18240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:25.190328 18317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:25.192682 18317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775: Bootstrap starting.
I20260812 03:52:25.193222 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 18240
I20260812 03:52:25.193317 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:25.193368 18317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:25.193508 17627 external_mini_cluster.cc:1428] Setting key 31527fdc8c4808279c5186e2e6100e0a
I20260812 03:52:25.193717 18317 log.cc:824] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:25.194598 18317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775: No bootstrap required, opened a new log
I20260812 03:52:25.197075 18317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00db0b68eef444b6a2786fac18a61775" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 43493 } }
I20260812 03:52:25.197254 18317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:25.197288 18317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00db0b68eef444b6a2786fac18a61775, State: Initialized, Role: FOLLOWER
I20260812 03:52:25.197412 18317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00db0b68eef444b6a2786fac18a61775" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 43493 } }
I20260812 03:52:25.197480 18317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:25.197515 18317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:25.197557 18317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:25.198212 18317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00db0b68eef444b6a2786fac18a61775" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 43493 } }
I20260812 03:52:25.198355 18317 leader_election.cc:302] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00db0b68eef444b6a2786fac18a61775; no voters:
I20260812 03:52:25.198721 18317 leader_election.cc:288] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:25.198755 18322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:25.198907 18322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [term 1 LEADER]: Becoming Leader. State: Replica: 00db0b68eef444b6a2786fac18a61775, State: Running, Role: LEADER
I20260812 03:52:25.199005 18322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00db0b68eef444b6a2786fac18a61775" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 43493 } }
I20260812 03:52:25.199070 18317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:25.199445 18322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00db0b68eef444b6a2786fac18a61775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00db0b68eef444b6a2786fac18a61775" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 43493 } } }
I20260812 03:52:25.199530 18322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:25.199687 18322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00db0b68eef444b6a2786fac18a61775. Latest consensus state: current_term: 1 leader_uuid: "00db0b68eef444b6a2786fac18a61775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00db0b68eef444b6a2786fac18a61775" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 43493 } } }
I20260812 03:52:25.199752 18322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:25.199785 18330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:25.200348 18330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:25.202364 18330 catalog_manager.cc:1347] Generated new cluster ID: 0b8ced22d5ce4126988f05a16facc5ea
I20260812 03:52:25.202425 18330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:25.206679 18321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.194053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.17.54.254:43493)
0812 03:52:25.194217 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.194229 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.194269 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:52:25.194875 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.194882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.194940 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.195057 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:25.195816 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.196746 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.197832 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.198822 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.198968 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.202014 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:25.202018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:25.202019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:25.202033 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:25.204262 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.204775 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.204778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.204779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.204897 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.205147 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.205148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.205149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.205534 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:25.206128 (+ 594us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.206218 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.206429 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260812 03:52:25.209980 18330 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:25.210759 18330 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:25.216313 18330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00db0b68eef444b6a2786fac18a61775: Generated new TSK 0
I20260812 03:52:25.216507 18330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:25.250415 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:43493
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:25.336127 18348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:25.336346 18348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:25.336372 18348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:25.337564 18348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:25.337605 18348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:25.337661 18348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:25.339357 18348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:43493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:25.339617 18348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:25.339854 18348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:25.345176 18357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:25.345324 18355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:25.345245 18348 server_base.cc:1034] running on GCE node
W20260812 03:52:25.345207 18354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:25.345741 18348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:25.346050 18348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:25.347216 18348 hybrid_clock.cc:648] HybridClock initialized: now 1786506745347214 us; error 36 us; skew 500 ppm
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:25.350449 18348 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:25.350939 18348 webserver.cc:460] Webserver started at http://127.17.54.193:39695/ using document root <none> and password file <none>
I20260812 03:52:25.351202 18348 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:25.351277 18348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:25.351461 18348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:25.352662 18348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "040d15cdcd544063b955cd879a011d20"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "5c67b44ff96271e33dbdec902eb2aa6f"
server_key_iv: "5b504bdef5f591be5b89d5a74aa7e52d"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.353083 18348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "040d15cdcd544063b955cd879a011d20"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "5c67b44ff96271e33dbdec902eb2aa6f"
server_key_iv: "5b504bdef5f591be5b89d5a74aa7e52d"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.354660 18348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:25.355613 18365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.355834 18348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:25.355953 18348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "040d15cdcd544063b955cd879a011d20"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "5c67b44ff96271e33dbdec902eb2aa6f"
server_key_iv: "5b504bdef5f591be5b89d5a74aa7e52d"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.356057 18348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:25.370839 18348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:25.373176 18348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:25.373313 18348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:25.373522 18348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:25.373803 18348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:25.373848 18348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.373873 18348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:25.373888 18348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.380442 18348 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:35163
I20260812 03:52:25.380522 18490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:35163 every 8 connection(s)
I20260812 03:52:25.380899 18348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:25.386612 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 18348
I20260812 03:52:25.386737 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:25.386960 17627 external_mini_cluster.cc:1428] Setting key 764d9e65d3485bc91797c6ba04988045
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:25.391920 18321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.381786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:37543 (local address 127.17.54.254:43493)
0812 03:52:25.381867 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.381871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.382108 (+ 237us) server_negotiation.cc:407] Connection header received
0812 03:52:25.383239 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.383240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.383252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.383313 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:25.383997 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.384587 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.385741 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.385971 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.386036 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.388761 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:25.388763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:25.388763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:25.388772 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:25.390453 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.390899 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.390901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.390901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.390966 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.391224 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.391225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.391226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.391352 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:25.391426 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.391728 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.391824 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:52:25.392129 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.381745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:43493 (local address 127.17.54.193:37543)
0812 03:52:25.381978 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:25.382019 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:25.383136 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:25.383383 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:25.383386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:25.383682 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:25.383892 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.383897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.384679 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.384680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.385646 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.385649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.386057 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.386058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.386171 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.386968 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:25.386977 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:25.388650 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:25.390558 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:25.390566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:25.390569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:25.390809 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:25.391035 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:25.391036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:25.391037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:25.391155 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:25.391430 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:25.391439 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:25.391586 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 03:52:25.391694 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:52:25.391880 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":115,"threads_started":1}
I20260812 03:52:25.392683 18491 heartbeater.cc:347] Connected to a master server at 127.17.54.254:43493
I20260812 03:52:25.392803 18491 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:25.392990 18491 heartbeater.cc:511] Master 127.17.54.254:43493 requested a full tablet report, sending...
I20260812 03:52:25.393601 18278 ts_manager.cc:194] Registered new tserver with Master: 040d15cdcd544063b955cd879a011d20 (127.17.54.193:35163)
I20260812 03:52:25.394213 18278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:37543
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:25.418092 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:43493
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:25.511026 18496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:25.511260 18496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:25.511287 18496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:25.512517 18496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:25.512558 18496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:25.512588 18496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:25.514441 18496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:43493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:25.514667 18496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:25.514895 18496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:25.520648 18501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:25.520668 18502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:25.520668 18496 server_base.cc:1034] running on GCE node
W20260812 03:52:25.520785 18504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:25.521091 18496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:25.521323 18496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:25.522464 18496 hybrid_clock.cc:648] HybridClock initialized: now 1786506745522430 us; error 55 us; skew 500 ppm
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:25.525161 18496 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:25.525596 18496 webserver.cc:460] Webserver started at http://127.17.54.194:33683/ using document root <none> and password file <none>
I20260812 03:52:25.525813 18496 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:25.525857 18496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:25.525949 18496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:25.527077 18496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "19be91d81a6074082ceb9626b92e80f7"
server_key_iv: "4274c766866b957be28ac786a789a01c"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.527393 18496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "19be91d81a6074082ceb9626b92e80f7"
server_key_iv: "4274c766866b957be28ac786a789a01c"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.528700 18496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:52:25.529479 18512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.529677 18496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:25.529788 18496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "19be91d81a6074082ceb9626b92e80f7"
server_key_iv: "4274c766866b957be28ac786a789a01c"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.529867 18496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:25.542135 18496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:25.544870 18496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:25.545071 18496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:25.545370 18496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:25.545740 18496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:25.545804 18496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.545845 18496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:25.545907 18496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.553100 18496 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:46197
I20260812 03:52:25.553145 18649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:46197 every 8 connection(s)
I20260812 03:52:25.553488 18496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:52:25.554121 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 18496
I20260812 03:52:25.554214 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:25.554461 17627 external_mini_cluster.cc:1428] Setting key 3394bbf2304a5e2206c1bc0c9304aadd
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
I20260812 03:52:25.565117 18321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.554332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:53855 (local address 127.17.54.254:43493)
0812 03:52:25.554388 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.554391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.554661 (+ 270us) server_negotiation.cc:407] Connection header received
0812 03:52:25.555971 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.555972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.555985 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.556048 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:25.556926 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.557434 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.559011 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.559235 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.559324 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.561953 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:25.561955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:25.561956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:25.561964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:25.563656 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.564064 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.564066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.564066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.564129 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.564381 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.564382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.564382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.564531 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:25.564602 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.564903 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.564994 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:52:25.565234 18653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.554293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:43493 (local address 127.17.54.194:53855)
0812 03:52:25.554494 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:25.554549 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:25.555820 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:25.556124 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:25.556128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:25.556532 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:25.556815 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.556821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.557532 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.557533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.558929 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.558932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.559333 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.559335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.559497 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.560127 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:25.560136 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:25.561852 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:25.563750 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:25.563756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:25.563758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:25.563984 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:25.564204 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:25.564205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:25.564205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:25.564304 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:25.564598 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:25.564602 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:25.564723 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 03:52:25.564853 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 03:52:25.565013 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260812 03:52:25.565703 18650 heartbeater.cc:347] Connected to a master server at 127.17.54.254:43493
I20260812 03:52:25.565817 18650 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:25.565989 18650 heartbeater.cc:511] Master 127.17.54.254:43493 requested a full tablet report, sending...
I20260812 03:52:25.566423 18278 ts_manager.cc:194] Registered new tserver with Master: 0f9e37a0d2f242e49a2f3699c88bdfcf (127.17.54.194:46197)
I20260812 03:52:25.566861 18278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:53855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:25.586673 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:43493
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:25.671419 18656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:25.671636 18656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:25.671660 18656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:25.672899 18656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:25.672942 18656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:25.672969 18656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:25.674651 18656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:43493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:25.674911 18656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:25.675148 18656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:25.680419 18662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:25.680498 18656 server_base.cc:1034] running on GCE node
W20260812 03:52:25.680428 18661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:25.680434 18664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:25.680898 18656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:25.681154 18656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:25.682349 18656 hybrid_clock.cc:648] HybridClock initialized: now 1786506745682341 us; error 37 us; skew 500 ppm
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:25.684985 18656 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:25.685474 18656 webserver.cc:460] Webserver started at http://127.17.54.195:41129/ using document root <none> and password file <none>
I20260812 03:52:25.685734 18656 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:25.685804 18656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:25.685958 18656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:25.687069 18656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "ff1f6eb155cd4ac9b63a6b94998546f9"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "439266b8a9ce80c482bb6528fc8c91ce"
server_key_iv: "987abecc89dab21efe0d6b81d4950427"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.687458 18656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "ff1f6eb155cd4ac9b63a6b94998546f9"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "439266b8a9ce80c482bb6528fc8c91ce"
server_key_iv: "987abecc89dab21efe0d6b81d4950427"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.688828 18656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:52:25.689639 18672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.689852 18656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:25.689958 18656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "ff1f6eb155cd4ac9b63a6b94998546f9"
format_stamp: "Formatted at 2026-08-12 03:52:25 on dist-test-slave-tm1g"
server_key: "439266b8a9ce80c482bb6528fc8c91ce"
server_key_iv: "987abecc89dab21efe0d6b81d4950427"
server_key_version: "encryptionkey@0"
I20260812 03:52:25.690033 18656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:25.708804 18656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:25.711334 18656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:25.711519 18656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:25.711781 18656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:25.712111 18656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:25.712167 18656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.712213 18656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:25.712239 18656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:25.718954 18656 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:35289
I20260812 03:52:25.719019 18809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:35289 every 8 connection(s)
I20260812 03:52:25.719377 18656 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:25.722764 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 18656
I20260812 03:52:25.722873 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:25.723060 17627 external_mini_cluster.cc:1428] Setting key 69b84c9283e4aaeea8914f02d6a6bbe4
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:25.729652 18321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.720281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:47895 (local address 127.17.54.254:43493)
0812 03:52:25.720351 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.720353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.720640 (+ 287us) server_negotiation.cc:407] Connection header received
0812 03:52:25.721561 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.721562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.721573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.721640 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:25.722280 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.722923 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.724360 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.724554 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.724613 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.726842 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:25.726844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:25.726844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:25.726851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:25.728266 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.728711 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.728713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.728713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.728773 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.729032 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.729033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.729034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.729159 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:25.729228 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.729519 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.729587 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:52:25.729838 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.720269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:43493 (local address 127.17.54.195:47895)
0812 03:52:25.720449 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:25.720513 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:25.721447 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:25.721695 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:25.721698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:25.721976 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:25.722177 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.722181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.723017 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.723019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.724277 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.724280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.724630 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.724631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.724740 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.725239 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:25.725248 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:25.726717 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:25.728381 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:25.728388 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:25.728389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:25.728633 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:25.728842 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:25.728843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:25.728843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:25.728946 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:25.729245 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:25.729251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:25.729397 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 03:52:25.729486 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:52:25.729614 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260812 03:52:25.730368 18810 heartbeater.cc:347] Connected to a master server at 127.17.54.254:43493
I20260812 03:52:25.730491 18810 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:25.730672 18810 heartbeater.cc:511] Master 127.17.54.254:43493 requested a full tablet report, sending...
I20260812 03:52:25.731106 18278 ts_manager.cc:194] Registered new tserver with Master: ff1f6eb155cd4ac9b63a6b94998546f9 (127.17.54.195:35289)
I20260812 03:52:25.731515 18278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:47895
I20260812 03:52:25.734818 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:52:25.743755 18321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.737057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43304 (local address 127.17.54.254:43493)
0812 03:52:25.737155 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.737157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.737261 (+ 104us) server_negotiation.cc:407] Connection header received
0812 03:52:25.737343 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.737344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.737354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.737400 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:25.737975 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.738546 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.739263 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.739459 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.739545 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.740782 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:25.740783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:25.740784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:25.740791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:25.742171 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.742673 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.742674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.742674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.742737 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.742986 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.742987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.742987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.743222 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:25.743353 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.743510 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.743617 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:52:25.748027 18278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:25.750650 18278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:25.759194 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.754233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.17.54.194:46197)
0812 03:52:25.754517 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.754523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.754536 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:52:25.754583 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.754586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.754605 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.754698 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:25.755142 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.756078 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.757973 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.758688 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.758794 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.758864 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.759055 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.759101 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":145,"threads_started":1}
I20260812 03:52:25.759198 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.754239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:35163 (local address 127.0.0.1:58626)
0812 03:52:25.754379 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:25.754389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:25.754435 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:25.754936 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:25.754937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:25.754940 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:25.755008 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.755010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.756571 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.756573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.757863 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.757865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.758867 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.758868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.758939 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.758944 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:25.758982 (+ 38us) client_negotiation.cc:240] Negotiation successful
0812 03:52:25.759064 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":7,"thread_start_us":117,"threads_started":1}
I20260812 03:52:25.759213 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.753973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:35289 (local address 127.0.0.1:38868)
0812 03:52:25.754169 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:25.754182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:25.754237 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:25.754975 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:25.754976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:25.754978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:25.755043 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.755045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.756673 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.756674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.757707 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.757710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.758859 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.758861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.758942 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.758947 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:25.758982 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 03:52:25.759138 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":133,"thread_start_us":143,"threads_started":1}
I20260812 03:52:25.759497 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.753907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:46197 (local address 127.0.0.1:60138)
0812 03:52:25.754059 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:25.754104 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:25.754182 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:25.754801 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:25.754804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:25.754807 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:25.754870 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.754872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.756194 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.756195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.757854 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.757859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.758793 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.758794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.758899 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.758916 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:52:25.759033 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:52:25.759087 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":35,"thread_start_us":102,"threads_started":1}
I20260812 03:52:25.759953 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.754503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38868 (local address 127.17.54.195:35289)
0812 03:52:25.754781 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.754786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.754796 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:25.754836 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.754838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.754855 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.754960 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:25.755120 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.756473 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.757806 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.758722 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.759035 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.759104 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.759133 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.759181 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":106,"threads_started":1}
I20260812 03:52:25.760015 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.754495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.17.54.193:35163)
0812 03:52:25.754704 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.754709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.754721 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:25.754768 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.754770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.754788 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.755092 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:25.755190 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.756429 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.757952 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.758743 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.759640 (+ 897us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.759722 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.759751 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.759808 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":129,"threads_started":1}
I20260812 03:52:25.760468 18571 tablet_service.cc:1467] Processing CreateTablet for tablet d3be9c7ca57b4b3fa0f6c0da5f9e15f7 (DEFAULT_TABLE table=test-table [id=4b25db2266164cd9932f9c8fdad25626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:25.760684 18421 tablet_service.cc:1467] Processing CreateTablet for tablet d3be9c7ca57b4b3fa0f6c0da5f9e15f7 (DEFAULT_TABLE table=test-table [id=4b25db2266164cd9932f9c8fdad25626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:25.760859 18571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3be9c7ca57b4b3fa0f6c0da5f9e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:25.761057 18421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3be9c7ca57b4b3fa0f6c0da5f9e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:25.763430 18735 tablet_service.cc:1467] Processing CreateTablet for tablet d3be9c7ca57b4b3fa0f6c0da5f9e15f7 (DEFAULT_TABLE table=test-table [id=4b25db2266164cd9932f9c8fdad25626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:25.763926 18735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3be9c7ca57b4b3fa0f6c0da5f9e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:25.765643 18828 tablet_bootstrap.cc:492] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf: Bootstrap starting.
I20260812 03:52:25.766912 18828 tablet_bootstrap.cc:654] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:25.768070 18828 log.cc:824] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:25.768656 18831 tablet_bootstrap.cc:492] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20: Bootstrap starting.
I20260812 03:52:25.769294 18828 tablet_bootstrap.cc:492] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf: No bootstrap required, opened a new log
I20260812 03:52:25.769443 18828 ts_tablet_manager.cc:1397] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 03:52:25.770452 18831 tablet_bootstrap.cc:654] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:25.770597 18833 tablet_bootstrap.cc:492] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9: Bootstrap starting.
I20260812 03:52:25.771926 18833 tablet_bootstrap.cc:654] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:25.772182 18831 log.cc:824] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:25.773229 18833 log.cc:824] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:25.773483 18831 tablet_bootstrap.cc:492] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20: No bootstrap required, opened a new log
I20260812 03:52:25.773578 18831 ts_tablet_manager.cc:1397] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 03:52:25.774135 18833 tablet_bootstrap.cc:492] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9: No bootstrap required, opened a new log
I20260812 03:52:25.774222 18833 ts_tablet_manager.cc:1397] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 03:52:25.774791 18828 raft_consensus.cc:348] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.775149 18828 raft_consensus.cc:374] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:25.775203 18828 raft_consensus.cc:729] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9e37a0d2f242e49a2f3699c88bdfcf, State: Initialized, Role: FOLLOWER
I20260812 03:52:25.775395 18828 consensus_queue.cc:260] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.775957 18828 ts_tablet_manager.cc:1428] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260812 03:52:25.776160 18650 heartbeater.cc:503] Master 127.17.54.254:43493 was elected leader, sending a full tablet report...
I20260812 03:52:25.776870 18833 raft_consensus.cc:348] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.777081 18833 raft_consensus.cc:374] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:25.777122 18833 raft_consensus.cc:729] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1f6eb155cd4ac9b63a6b94998546f9, State: Initialized, Role: FOLLOWER
I20260812 03:52:25.777261 18833 consensus_queue.cc:260] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.777387 18831 raft_consensus.cc:348] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.777621 18833 ts_tablet_manager.cc:1428] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:52:25.777652 18831 raft_consensus.cc:374] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:25.777688 18810 heartbeater.cc:503] Master 127.17.54.254:43493 was elected leader, sending a full tablet report...
I20260812 03:52:25.777701 18831 raft_consensus.cc:729] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040d15cdcd544063b955cd879a011d20, State: Initialized, Role: FOLLOWER
I20260812 03:52:25.777830 18831 consensus_queue.cc:260] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.778257 18831 ts_tablet_manager.cc:1428] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260812 03:52:25.778324 18491 heartbeater.cc:503] Master 127.17.54.254:43493 was elected leader, sending a full tablet report...
W20260812 03:52:25.804157 18651 tablet.cc:2369] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:52:25.881775 18492 tablet.cc:2369] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:25.952251 18838 raft_consensus.cc:482] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:25.952390 18838 raft_consensus.cc:504] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.953171 18838 leader_election.cc:288] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f9e37a0d2f242e49a2f3699c88bdfcf (127.17.54.194:46197), 040d15cdcd544063b955cd879a011d20 (127.17.54.193:35163)
I20260812 03:52:25.956223 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.953389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:46197 (local address 127.17.54.195:34283)
0812 03:52:25.953459 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:25.953473 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:25.953525 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:25.953697 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:25.953698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:25.953700 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:25.953786 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.953788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.954509 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.954511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.955320 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.955322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.955906 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.955907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.956035 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.956038 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:25.956111 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:52:25.956153 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:52:25.956223 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.953481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:34283 (local address 127.17.54.194:46197)
0812 03:52:25.953529 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.953531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.953541 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:25.953585 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.953586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.953609 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.953683 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:25.953866 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.954417 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.955412 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.955824 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.955910 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.955927 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.956118 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.956158 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:52:25.956622 18593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3be9c7ca57b4b3fa0f6c0da5f9e15f7" candidate_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" is_pre_election: true
I20260812 03:52:25.956681 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.953523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:37633 (local address 127.17.54.193:35163)
0812 03:52:25.953574 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.953577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.953684 (+ 107us) server_negotiation.cc:407] Connection header received
0812 03:52:25.953775 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.953776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.953788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.953869 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:25.954046 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.954576 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.955723 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.956083 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.956219 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.956229 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.956579 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.956617 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:52:25.956727 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.953414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:35163 (local address 127.17.54.195:37633)
0812 03:52:25.953601 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:25.953618 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:25.953698 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:25.953875 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:25.953875 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:25.953877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:25.953963 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.953965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.954667 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.954668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.955632 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:25.955635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.956336 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.956338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:25.956485 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.956490 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:25.956572 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:52:25.956612 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":112,"threads_started":1}
I20260812 03:52:25.956859 18593 raft_consensus.cc:2454] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff1f6eb155cd4ac9b63a6b94998546f9 in term 0.
I20260812 03:52:25.957108 18443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3be9c7ca57b4b3fa0f6c0da5f9e15f7" candidate_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040d15cdcd544063b955cd879a011d20" is_pre_election: true
I20260812 03:52:25.957234 18677 leader_election.cc:302] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f9e37a0d2f242e49a2f3699c88bdfcf, ff1f6eb155cd4ac9b63a6b94998546f9; no voters:
I20260812 03:52:25.957267 18443 raft_consensus.cc:2454] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff1f6eb155cd4ac9b63a6b94998546f9 in term 0.
I20260812 03:52:25.957384 18838 raft_consensus.cc:2781] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:25.957451 18838 raft_consensus.cc:482] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:25.957485 18838 raft_consensus.cc:3037] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:25.958428 18838 raft_consensus.cc:504] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.958581 18838 leader_election.cc:288] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [CANDIDATE]: Term 1 election: Requested vote from peers 0f9e37a0d2f242e49a2f3699c88bdfcf (127.17.54.194:46197), 040d15cdcd544063b955cd879a011d20 (127.17.54.193:35163)
I20260812 03:52:25.958736 18593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3be9c7ca57b4b3fa0f6c0da5f9e15f7" candidate_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf"
I20260812 03:52:25.958779 18443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3be9c7ca57b4b3fa0f6c0da5f9e15f7" candidate_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040d15cdcd544063b955cd879a011d20"
I20260812 03:52:25.958837 18593 raft_consensus.cc:3037] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:25.958873 18443 raft_consensus.cc:3037] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:25.959704 18443 raft_consensus.cc:2454] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 040d15cdcd544063b955cd879a011d20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff1f6eb155cd4ac9b63a6b94998546f9 in term 1.
I20260812 03:52:25.959888 18676 leader_election.cc:302] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 040d15cdcd544063b955cd879a011d20, ff1f6eb155cd4ac9b63a6b94998546f9; no voters:
I20260812 03:52:25.959910 18593 raft_consensus.cc:2454] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P 0f9e37a0d2f242e49a2f3699c88bdfcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff1f6eb155cd4ac9b63a6b94998546f9 in term 1.
I20260812 03:52:25.960021 18838 raft_consensus.cc:2781] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:25.960201 18838 raft_consensus.cc:686] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [term 1 LEADER]: Becoming Leader. State: Replica: ff1f6eb155cd4ac9b63a6b94998546f9, State: Running, Role: LEADER
I20260812 03:52:25.960300 18838 consensus_queue.cc:237] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } }
I20260812 03:52:25.961292 18277 catalog_manager.cc:5495] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff1f6eb155cd4ac9b63a6b94998546f9 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9e37a0d2f242e49a2f3699c88bdfcf" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff1f6eb155cd4ac9b63a6b94998546f9" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "040d15cdcd544063b955cd879a011d20" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 35163 } health_report { overall_health: UNKNOWN } } }
W20260812 03:52:25.969978 18811 tablet.cc:2369] T d3be9c7ca57b4b3fa0f6c0da5f9e15f7 P ff1f6eb155cd4ac9b63a6b94998546f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:25.989883 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:25.982047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.17.54.193:35163)
0812 03:52:25.982129 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:25.982132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:25.982251 (+ 119us) server_negotiation.cc:407] Connection header received
0812 03:52:25.982369 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:25.982369 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:25.982388 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:25.982455 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:25.983058 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.983558 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.984300 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:25.984499 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:25.984575 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:25.987013 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:25.987029 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:25.987030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:25.987037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:25.988593 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.989008 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.989013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.989013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.989088 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:25.989299 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:25.989300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:25.989300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:25.989468 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:25.989641 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:25.989679 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 03:52:25.989766 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 03:52:25 dist-test-slave-tm1g krb5kdc[18225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:52:26 dist-test-slave-tm1g krb5kdc[18225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:26.016192 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:26.008009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58652 (local address 127.17.54.193:35163)
0812 03:52:26.008097 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:26.008099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:26.008140 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:52:26.008207 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:26.008208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:26.008218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:26.008275 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:26.008989 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:26.009507 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:26.010296 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:26.010607 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:26.010675 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:26.013287 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:26.013288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:26.013289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:26.013296 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:26.014838 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:26.015195 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:26.015196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:26.015196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:26.015251 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:26.015480 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:26.015481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:26.015481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:26.015605 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:26.015692 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:26.015937 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 03:52:26.016033 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 03:52:26 dist-test-slave-tm1g krb5kdc[18225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:52:26 dist-test-slave-tm1g krb5kdc[18225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:26.041374 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:26.033557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.17.54.193:35163)
0812 03:52:26.033616 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:26.033619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:26.033759 (+ 140us) server_negotiation.cc:407] Connection header received
0812 03:52:26.033837 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:26.033838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:26.033855 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:26.033939 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:26.034636 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:26.035203 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:26.036192 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:26.036381 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:26.036489 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:26.038961 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:26.038963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:26.038963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:26.038971 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:26.040438 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:26.040768 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:26.040769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:26.040769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:26.040821 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:26.040987 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:26.040988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:26.040988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:26.041084 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:26.041161 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:26.041245 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:52:26.041305 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260812 03:52:26.042146 18396 tablet_service.cc:2908] Rejecting scan request for tablet d3be9c7ca57b4b3fa0f6c0da5f9e15f7: Uninitialized: clean time has not yet been initialized
I20260812 03:52:26.043414 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 18348
I20260812 03:52:26.051707 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 18496
I20260812 03:52:26.061528 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 18656
I20260812 03:52:26.071698 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 18240
2026-08-12T03:52:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2280 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:26 dist-test-slave-tm1g krb5kdc[18858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:26 dist-test-slave-tm1g krb5kdc[18858](info): set up 2 sockets
Aug 12 03:52:26 dist-test-slave-tm1g krb5kdc[18858](info): commencing operation
krb5kdc: starting...
W20260812 03:52:27.209136 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:27 dist-test-slave-tm1g krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:27.354772 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:41647
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:41647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:27.445717 18872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:27.446020 18872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:27.446065 18872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:27.447495 18872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:27.447587 18872 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:52:27.447610 18872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:27.447634 18872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:27.447664 18872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:27.447695 18872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:27.449388 18872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:41647
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:41647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:27.449714 18872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:27.449987 18872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:27.455335 18881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:27.455540 18872 server_base.cc:1034] running on GCE node
W20260812 03:52:27.455335 18877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:27.455366 18878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:27.456022 18872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:27.456349 18872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:27.457542 18872 hybrid_clock.cc:648] HybridClock initialized: now 1786506747457530 us; error 50 us; skew 500 ppm
Aug 12 03:52:27 dist-test-slave-tm1g krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:27.460489 18872 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:27.460999 18872 webserver.cc:460] Webserver started at http://127.17.54.254:42569/ using document root <none> and password file <none>
I20260812 03:52:27.461246 18872 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:27.461318 18872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:27.461475 18872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:27.462675 18872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "43031759fedd45e690261fa33331a9e2"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "1501508d72c42af3c5f5655241b0d399"
server_key_iv: "c8fab1cda672ad09c338ad714576235c"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.463002 18872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "43031759fedd45e690261fa33331a9e2"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "1501508d72c42af3c5f5655241b0d399"
server_key_iv: "c8fab1cda672ad09c338ad714576235c"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.464479 18872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:52:27.465418 18890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:27.465678 18872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:27.465808 18872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "43031759fedd45e690261fa33331a9e2"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "1501508d72c42af3c5f5655241b0d399"
server_key_iv: "c8fab1cda672ad09c338ad714576235c"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.465914 18872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:27.488916 18872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:27.491895 18872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:27.492094 18872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:27.498138 18872 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:41647
I20260812 03:52:27.498191 18956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:41647 every 8 connection(s)
I20260812 03:52:27.498602 18872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:27.499366 18957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:27.501881 18957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2: Bootstrap starting.
I20260812 03:52:27.502180 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 18872
I20260812 03:52:27.502363 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:27.502542 17627 external_mini_cluster.cc:1428] Setting key 3f2b7aa758ee00d9efdf4f786b9af9b3
I20260812 03:52:27.502668 18957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:27.502997 18957 log.cc:824] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:27.503777 18957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2: No bootstrap required, opened a new log
I20260812 03:52:27.505437 18957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43031759fedd45e690261fa33331a9e2" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41647 } }
I20260812 03:52:27.505604 18957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:27.505627 18957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43031759fedd45e690261fa33331a9e2, State: Initialized, Role: FOLLOWER
I20260812 03:52:27.505774 18957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43031759fedd45e690261fa33331a9e2" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41647 } }
I20260812 03:52:27.505852 18957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:27.505877 18957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:27.505945 18957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:27.506636 18957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43031759fedd45e690261fa33331a9e2" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41647 } }
I20260812 03:52:27.506765 18957 leader_election.cc:302] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43031759fedd45e690261fa33331a9e2; no voters:
I20260812 03:52:27.506975 18957 leader_election.cc:288] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:27.507011 18962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:27.507129 18962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [term 1 LEADER]: Becoming Leader. State: Replica: 43031759fedd45e690261fa33331a9e2, State: Running, Role: LEADER
I20260812 03:52:27.507272 18957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:27.507279 18962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43031759fedd45e690261fa33331a9e2" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41647 } }
I20260812 03:52:27.507759 18964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43031759fedd45e690261fa33331a9e2. Latest consensus state: current_term: 1 leader_uuid: "43031759fedd45e690261fa33331a9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43031759fedd45e690261fa33331a9e2" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41647 } } }
I20260812 03:52:27.507921 18964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:27.508134 18963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43031759fedd45e690261fa33331a9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43031759fedd45e690261fa33331a9e2" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41647 } } }
I20260812 03:52:27.508252 18963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2 [sys.catalog]: This master's current role is: LEADER
W20260812 03:52:27.509439 18982 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:27.509505 18982 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:27.509613 18973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:27.509824 18973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:52:27 dist-test-slave-tm1g krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:27.511649 18973 catalog_manager.cc:1347] Generated new cluster ID: e35f7db2ac1b49eda08ec265cf95de09
I20260812 03:52:27.511705 18973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:27.515659 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.503197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.17.54.254:41647)
0812 03:52:27.503357 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:27.503367 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:27.503404 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:52:27.503944 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:27.503952 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:27.504012 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:27.504167 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:27.504881 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.506530 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.507565 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.508212 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.508312 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.511691 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:27.511695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:27.511696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:27.511708 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:27.513800 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.514290 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.514294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.514294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.514447 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.514696 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.514697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.514698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.515021 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:27.515191 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:27.515272 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 03:52:27.515416 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260812 03:52:27.524137 18973 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:27.524688 18973 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:27.530537 18973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2: Generated new TSK 0
I20260812 03:52:27.530763 18973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:27.559587 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41647
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:27.645231 18988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:27.645462 18988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:27.645488 18988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:27.646713 18988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:27.646755 18988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:27.646775 18988 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:27.646793 18988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:27.648365 18988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:41647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:27.648571 18988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:27.648762 18988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:27.654217 18998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:27.654271 18996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:27.654284 18988 server_base.cc:1034] running on GCE node
W20260812 03:52:27.654238 18995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:27.654721 18988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:27.654992 18988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:27.656178 18988 hybrid_clock.cc:648] HybridClock initialized: now 1786506747656145 us; error 52 us; skew 500 ppm
Aug 12 03:52:27 dist-test-slave-tm1g krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:27.658939 18988 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:27.659408 18988 webserver.cc:460] Webserver started at http://127.17.54.193:36477/ using document root <none> and password file <none>
I20260812 03:52:27.659674 18988 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:27.659749 18988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:27.659909 18988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:27.661051 18988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "b449d9c8ae694a3faba368b1d8777db8"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "b8281f47dbda9b2f2cacf89cb533aa70"
server_key_iv: "b75c57ed68c88a254bd6f09df5701e45"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.661458 18988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "b449d9c8ae694a3faba368b1d8777db8"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "b8281f47dbda9b2f2cacf89cb533aa70"
server_key_iv: "b75c57ed68c88a254bd6f09df5701e45"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.663035 18988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:27.664109 19006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:27.664348 18988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:27.664506 18988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "b449d9c8ae694a3faba368b1d8777db8"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "b8281f47dbda9b2f2cacf89cb533aa70"
server_key_iv: "b75c57ed68c88a254bd6f09df5701e45"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.664592 18988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:27.676034 18988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:27.678545 18988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:27.678730 18988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:27.679344 18988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:27.679406 18988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:27.679442 18988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:27.679477 18988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:52:27 dist-test-slave-tm1g krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:27.687115 18988 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:40503
I20260812 03:52:27.687505 18988 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:27.687526 19136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:40503 every 8 connection(s)
I20260812 03:52:27.690039 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.680036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:47991 (local address 127.17.54.254:41647)
0812 03:52:27.680120 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:27.680123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:27.680281 (+ 158us) server_negotiation.cc:407] Connection header received
0812 03:52:27.680794 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:27.680796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:27.680813 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:27.680898 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:27.681877 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.682810 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.683917 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.684286 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.684381 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.686885 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:27.686887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:27.686887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:27.686895 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:27.688563 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.689003 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.689004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.689005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.689065 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.689338 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.689339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.689339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.689469 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:27.689547 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:27.689841 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 03:52:27.689946 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:52:27.690220 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.679949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41647 (local address 127.17.54.193:47991)
0812 03:52:27.680140 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:27.680181 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:27.680675 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:27.680972 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:27.680976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:27.681437 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:27.681748 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:27.681756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.682930 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.682931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:27.683841 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:27.683843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.684428 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.684430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:27.684570 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.685129 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:27.685141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:27.686775 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:27.688670 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:27.688678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:27.688680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:27.688919 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:27.689147 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:27.689147 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:27.689148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:27.689263 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:27.689539 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:27.689547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:27.689698 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 03:52:27.689805 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:52:27.689979 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260812 03:52:27.690766 19137 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41647
I20260812 03:52:27.690937 19137 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:27.691130 19137 heartbeater.cc:511] Master 127.17.54.254:41647 requested a full tablet report, sending...
I20260812 03:52:27.691694 18906 ts_manager.cc:194] Registered new tserver with Master: b449d9c8ae694a3faba368b1d8777db8 (127.17.54.193:40503)
I20260812 03:52:27.692700 18906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:47991
I20260812 03:52:27.695638 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 18988
I20260812 03:52:27.695750 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:27.695914 17627 external_mini_cluster.cc:1428] Setting key 9202356df1f0b1050686d2b69f19805a
I20260812 03:52:27.698172 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.691936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:56241 (local address 127.17.54.254:41647)
0812 03:52:27.691981 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:27.691982 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:27.692091 (+ 109us) server_negotiation.cc:407] Connection header received
0812 03:52:27.692165 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:27.692165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:27.692173 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:27.692216 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:27.692564 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.693091 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.693960 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.694153 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.694238 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.695294 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:27.695296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:27.695296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:27.695304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:27.697022 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.697365 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.697366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.697367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.697424 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.697627 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.697628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.697628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.697741 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:27.697812 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:27.697992 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 03:52:27.698073 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:27.698184 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.691872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41647 (local address 127.17.54.193:56241)
0812 03:52:27.692030 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:27.692042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:27.692108 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:27.692223 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:27.692224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:27.692430 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:27.692499 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:27.692501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.693189 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.693190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:27.693813 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:27.693816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.694240 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.694241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:27.694356 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.694816 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:27.694821 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:27.695206 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:27.697105 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:27.697106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:27.697106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:27.697290 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:27.697486 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:27.697486 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:27.697487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:27.697554 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:27.697815 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:27.697816 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:27.697878 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 03:52:27.697962 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:52:27.698076 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":47,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:27.726068 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41647
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:27.816042 19147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:27.816265 19147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:27.816291 19147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:27.817503 19147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:27.817548 19147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:27.817569 19147 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:27.817587 19147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:27.819172 19147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:41647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:27.819451 19147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:27.819705 19147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:27.825232 19156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:27.825248 19152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:27.825346 19147 server_base.cc:1034] running on GCE node
W20260812 03:52:27.825345 19153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:27.825980 19147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:27.826225 19147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:27.827416 19147 hybrid_clock.cc:648] HybridClock initialized: now 1786506747827401 us; error 39 us; skew 500 ppm
Aug 12 03:52:27 dist-test-slave-tm1g krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:27.830101 19147 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:27.830600 19147 webserver.cc:460] Webserver started at http://127.17.54.194:40897/ using document root <none> and password file <none>
I20260812 03:52:27.830821 19147 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:27.830863 19147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:27.830955 19147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:27.832068 19147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "4bb1f2101f254b94b88e969c1cb8047e"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "60ae502f8c899de28f8c07fac0948ade"
server_key_iv: "b5831ec4feab02435c1a76ba0f4211d9"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.832386 19147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "4bb1f2101f254b94b88e969c1cb8047e"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "60ae502f8c899de28f8c07fac0948ade"
server_key_iv: "b5831ec4feab02435c1a76ba0f4211d9"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.833709 19147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:27.834493 19163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:27.834659 19147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:27.834748 19147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "4bb1f2101f254b94b88e969c1cb8047e"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "60ae502f8c899de28f8c07fac0948ade"
server_key_iv: "b5831ec4feab02435c1a76ba0f4211d9"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.834805 19147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:27.846973 19147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:27.849349 19147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:27.849467 19147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:27.849959 19147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:27.849996 19147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:27.850019 19147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:27.850032 19147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:27.858016 19147 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:34551
I20260812 03:52:27.858091 19292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:34551 every 8 connection(s)
I20260812 03:52:27.858497 19147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:52:27 dist-test-slave-tm1g krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:27.861824 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.850725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:36993 (local address 127.17.54.254:41647)
0812 03:52:27.850790 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:27.850793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:27.851032 (+ 239us) server_negotiation.cc:407] Connection header received
0812 03:52:27.851770 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:27.851771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:27.851785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:27.851856 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:27.852921 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.853829 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.855675 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.856018 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.856100 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.859115 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:27.859117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:27.859117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:27.859124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:27.860402 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.860866 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.860868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.860868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.860933 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.861169 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.861170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.861170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.861283 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:27.861374 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:27.861665 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 03:52:27.861755 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:52:27.861899 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.850701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41647 (local address 127.17.54.194:36993)
0812 03:52:27.850898 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:27.850938 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:27.851630 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:27.851931 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:27.851935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:27.852268 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:27.852797 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:27.852805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.853962 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.853964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:27.855541 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:27.855546 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.856119 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.856120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:27.856281 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.856984 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:27.856995 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:27.859033 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:27.860485 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:27.860492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:27.860494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:27.860762 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:27.861003 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:27.861004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:27.861004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:27.861108 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:27.861361 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:27.861364 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:27.861475 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 03:52:27.861573 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:52:27.861705 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260812 03:52:27.862172 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 19147
I20260812 03:52:27.862273 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:27.862507 17627 external_mini_cluster.cc:1428] Setting key 4a847a05a6a3b7c8a5a62dd0eabea0f4
I20260812 03:52:27.862561 19293 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41647
I20260812 03:52:27.862655 19293 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:27.862852 19293 heartbeater.cc:511] Master 127.17.54.254:41647 requested a full tablet report, sending...
I20260812 03:52:27.863318 18908 ts_manager.cc:194] Registered new tserver with Master: 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551)
I20260812 03:52:27.863837 18908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:36993
I20260812 03:52:27.872982 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.863717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41647 (local address 127.17.54.194:48455)
0812 03:52:27.864073 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:27.864084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:27.864821 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:27.866119 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:27.866120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:27.866381 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:27.866459 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:27.866461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.867349 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.867350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:27.867984 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:27.867986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.868577 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.868578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:27.868683 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.869185 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:27.869190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:27.869722 (+ 532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:27.871704 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:27.871705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:27.871706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:27.871887 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:27.872154 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:27.872155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:27.872155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:27.872224 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:27.872657 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:27.872658 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:27.872714 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 03:52:27.872788 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:52:27.872902 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":84,"threads_started":1}
I20260812 03:52:27.873078 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:27.863865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:48455 (local address 127.17.54.254:41647)
0812 03:52:27.863931 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:27.863933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:27.865969 (+ 2036us) server_negotiation.cc:407] Connection header received
0812 03:52:27.865994 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:27.865994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:27.866005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:27.866095 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:27.866543 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.867216 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.868069 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:27.868499 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:27.868610 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:27.869810 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:27.869812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:27.869813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:27.869822 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:27.871605 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.871970 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.871971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.871971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.872043 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:27.872313 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:27.872314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:27.872315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:27.872554 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:27.872648 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:27.872841 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 03:52:27.872950 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:27.893533 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41647
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:27.982656 19302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:27.982864 19302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:27.982889 19302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:27.984015 19302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:27.984050 19302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:27.984071 19302 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:27.984087 19302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:27.985522 19302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:41647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:27.985728 19302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:27.985906 19302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:27.991014 19312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:27.991063 19308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:27.991133 19302 server_base.cc:1034] running on GCE node
W20260812 03:52:27.991251 19309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:27.991457 19302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:27.991729 19302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:27.992885 19302 hybrid_clock.cc:648] HybridClock initialized: now 1786506747992861 us; error 35 us; skew 500 ppm
Aug 12 03:52:27 dist-test-slave-tm1g krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:27.995519 19302 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:27.996001 19302 webserver.cc:460] Webserver started at http://127.17.54.195:44789/ using document root <none> and password file <none>
I20260812 03:52:27.996253 19302 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:27.996321 19302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:27.996474 19302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:27.997546 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "feda85fe092746c4a805c5b7921686f5"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "5c5447c3f6b77e409a0a39089622d09d"
server_key_iv: "1a3b06f6d3aeb5fc50f6e5164d59d249"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.997916 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "feda85fe092746c4a805c5b7921686f5"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "5c5447c3f6b77e409a0a39089622d09d"
server_key_iv: "1a3b06f6d3aeb5fc50f6e5164d59d249"
server_key_version: "encryptionkey@0"
I20260812 03:52:27.999403 19302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:28.000262 19319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:28.000461 19302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:28.000591 19302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "feda85fe092746c4a805c5b7921686f5"
format_stamp: "Formatted at 2026-08-12 03:52:27 on dist-test-slave-tm1g"
server_key: "5c5447c3f6b77e409a0a39089622d09d"
server_key_iv: "1a3b06f6d3aeb5fc50f6e5164d59d249"
server_key_version: "encryptionkey@0"
I20260812 03:52:28.000665 19302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:28.028296 19302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:28.030989 19302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:28.031176 19302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:28.031769 19302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:28.031811 19302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:28.031836 19302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:28.031848 19302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:52:28 dist-test-slave-tm1g krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:28.039785 19302 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:40507
I20260812 03:52:28.039909 19455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:40507 every 8 connection(s)
I20260812 03:52:28.040221 19302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:28.042690 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.032286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:37757 (local address 127.17.54.254:41647)
0812 03:52:28.032364 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.032367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.032659 (+ 292us) server_negotiation.cc:407] Connection header received
0812 03:52:28.033479 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.033481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.033497 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.033574 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:28.034574 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.035467 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.036592 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.036812 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.036895 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.039391 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:28.039393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:28.039393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:28.039403 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:28.041106 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.041564 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.041565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.041566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.041634 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.041922 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.041923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.041923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.042054 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:28.042136 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.042476 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.042585 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:52:28.042922 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.032263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41647 (local address 127.17.54.195:37757)
0812 03:52:28.032483 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.032540 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.033309 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.033700 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.033703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.034161 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:28.034486 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.034492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.035598 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.035599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.036484 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.036487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.036928 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.036929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.037074 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.037716 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:28.037725 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:28.039279 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:28.041205 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:28.041211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:28.041214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:28.041479 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:28.041709 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:28.041710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:28.041711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:28.041841 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:28.042126 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:28.042133 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:28.042283 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.042456 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.042669 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":122,"threads_started":1}
I20260812 03:52:28.043479 19456 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41647
I20260812 03:52:28.043589 19456 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:28.043777 19456 heartbeater.cc:511] Master 127.17.54.254:41647 requested a full tablet report, sending...
I20260812 03:52:28.044214 18906 ts_manager.cc:194] Registered new tserver with Master: feda85fe092746c4a805c5b7921686f5 (127.17.54.195:40507)
I20260812 03:52:28.044735 18906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:37757
I20260812 03:52:28.049966 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 19302
I20260812 03:52:28.050042 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:28.050227 17627 external_mini_cluster.cc:1428] Setting key 767e6de9dc9d546ab0201322bc08fab7
I20260812 03:52:28.050896 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.044681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:56629 (local address 127.17.54.254:41647)
0812 03:52:28.044728 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.044730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.044793 (+ 63us) server_negotiation.cc:407] Connection header received
0812 03:52:28.044868 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.044868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.044878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.044936 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:28.045301 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.045811 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.046642 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.046839 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.046935 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.047956 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:28.047958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:28.047958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:28.047965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:28.049304 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.049728 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.049729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.049729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.049794 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.050109 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.050111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.050111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.050251 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:28.050372 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.050687 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.050777 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:52:28.050976 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.044605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41647 (local address 127.17.54.195:56629)
0812 03:52:28.044739 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.044751 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.044810 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.044940 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.044941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.045130 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:28.045201 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.045203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.045919 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.045920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.046560 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.046562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.046932 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.046933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.047011 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.047519 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:28.047524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:28.047834 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:28.049463 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:28.049464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:28.049464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:28.049649 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:28.049900 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:28.049901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:28.049901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:28.049979 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:28.050396 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:28.050398 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:28.050477 (+ 79us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.050647 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.050819 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260812 03:52:28.051291 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:52:28 dist-test-slave-tm1g krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506748, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:52:28 dist-test-slave-tm1g krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506748, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:28.077389 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.069595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.17.54.254:41647)
0812 03:52:28.069650 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.069652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.069660 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:28.069701 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.069702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.069711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.069758 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:28.070495 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.071031 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.071816 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.072029 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.072086 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.074654 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:28.074656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:28.074656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:28.074664 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:28.076087 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.076514 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.076516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.076516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.076578 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.076801 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.076802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.076802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.076943 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:28.077035 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.077219 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.077311 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:52:28.079392 18906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:28.081076 18906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:28.088457 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.084067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.17.54.193:40503)
0812 03:52:28.084655 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.084659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.084669 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:28.084717 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.084719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.084732 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.084824 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:28.084976 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.086158 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.087419 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.087976 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.088079 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.088141 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.088289 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.088332 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":84,"threads_started":1}
I20260812 03:52:28.088465 19478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.083760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:40503 (local address 127.0.0.1:53452)
0812 03:52:28.084116 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.084125 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.084168 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.084824 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.084825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.084827 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:28.084877 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.084879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.086248 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.086249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.087331 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.087333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.088078 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.088079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.088163 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.088177 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.088273 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.088333 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260812 03:52:28.088476 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.083755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:34551 (local address 127.0.0.1:41026)
0812 03:52:28.083931 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.083973 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.084061 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.084388 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.084392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.084395 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:28.084477 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.084479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.086069 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.086070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.087079 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.087082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.088198 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.088199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.088257 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.088260 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.088330 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.088364 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":2,"thread_start_us":65,"threads_started":1}
I20260812 03:52:28.088476 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.083760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:40507 (local address 127.0.0.1:60574)
0812 03:52:28.084240 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.084249 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.084286 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.084933 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.084934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.084936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:28.084970 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.084970 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.085853 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.085854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.087173 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.087176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.088173 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.088174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.088255 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.088263 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.088295 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.088350 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":42,"spinlock_wait_cycles":896,"thread_start_us":128,"threads_started":1}
I20260812 03:52:28.089015 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.083853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.17.54.194:34551)
0812 03:52:28.084051 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.084056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.084069 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:52:28.084195 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.084197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.084216 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.084300 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:28.084657 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.085959 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.087186 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.088011 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.088096 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.088165 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.088692 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.088740 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
I20260812 03:52:28.089182 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.084052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.17.54.195:40507)
0812 03:52:28.084412 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.084416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.084426 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:28.084466 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.084468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.084482 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.085046 (+ 564us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:28.085100 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.085754 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.087257 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.088066 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.088524 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.088771 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.089000 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.089060 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":96,"threads_started":1}
I20260812 03:52:28.089283 19065 tablet_service.cc:1467] Processing CreateTablet for tablet 18face1d32e1480faea923f43361f8f8 (DEFAULT_TABLE table=test-table [id=c57dd9dbaf864e70abb5f4f533f04684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:28.089735 19065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18face1d32e1480faea923f43361f8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.089874 19374 tablet_service.cc:1467] Processing CreateTablet for tablet 18face1d32e1480faea923f43361f8f8 (DEFAULT_TABLE table=test-table [id=c57dd9dbaf864e70abb5f4f533f04684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:28.090258 19374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18face1d32e1480faea923f43361f8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.093611 19484 tablet_bootstrap.cc:492] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: Bootstrap starting.
I20260812 03:52:28.094475 19485 tablet_bootstrap.cc:492] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: Bootstrap starting.
I20260812 03:52:28.094635 19219 tablet_service.cc:1467] Processing CreateTablet for tablet 18face1d32e1480faea923f43361f8f8 (DEFAULT_TABLE table=test-table [id=c57dd9dbaf864e70abb5f4f533f04684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:28.094852 19484 tablet_bootstrap.cc:654] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.094987 19485 tablet_bootstrap.cc:654] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.095047 19219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18face1d32e1480faea923f43361f8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.095224 19485 log.cc:824] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:28.095228 19484 log.cc:824] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:28.096110 19484 tablet_bootstrap.cc:492] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: No bootstrap required, opened a new log
I20260812 03:52:28.096138 19485 tablet_bootstrap.cc:492] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: No bootstrap required, opened a new log
I20260812 03:52:28.096184 19485 ts_tablet_manager.cc:1397] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:28.096179 19484 ts_tablet_manager.cc:1397] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:52:28.097800 19485 raft_consensus.cc:348] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.097925 19485 raft_consensus.cc:374] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.097946 19485 raft_consensus.cc:729] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b449d9c8ae694a3faba368b1d8777db8, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.098076 19485 consensus_queue.cc:260] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.098388 19485 ts_tablet_manager.cc:1428] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:52:28.098433 19137 heartbeater.cc:503] Master 127.17.54.254:41647 was elected leader, sending a full tablet report...
I20260812 03:52:28.098976 19484 raft_consensus.cc:348] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.099148 19484 raft_consensus.cc:374] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.099193 19484 raft_consensus.cc:729] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feda85fe092746c4a805c5b7921686f5, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.099324 19484 consensus_queue.cc:260] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.099592 19484 ts_tablet_manager.cc:1428] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260812 03:52:28.099522 19491 tablet_bootstrap.cc:492] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Bootstrap starting.
I20260812 03:52:28.099700 19456 heartbeater.cc:503] Master 127.17.54.254:41647 was elected leader, sending a full tablet report...
I20260812 03:52:28.100311 19491 tablet_bootstrap.cc:654] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.100685 19491 log.cc:824] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:28.101368 19491 tablet_bootstrap.cc:492] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: No bootstrap required, opened a new log
I20260812 03:52:28.101454 19491 ts_tablet_manager.cc:1397] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:28.102947 19491 raft_consensus.cc:348] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.103092 19491 raft_consensus.cc:374] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.103138 19491 raft_consensus.cc:729] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb1f2101f254b94b88e969c1cb8047e, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.103253 19491 consensus_queue.cc:260] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.103528 19491 ts_tablet_manager.cc:1428] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:52:28.103585 19293 heartbeater.cc:503] Master 127.17.54.254:41647 was elected leader, sending a full tablet report...
W20260812 03:52:28.109071 19294 tablet.cc:2369] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:28.132963 19492 raft_consensus.cc:482] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:28.133095 19492 raft_consensus.cc:504] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.133550 19492 leader_election.cc:288] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b449d9c8ae694a3faba368b1d8777db8 (127.17.54.193:40503), 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551)
I20260812 03:52:28.136749 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.133848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:60953 (local address 127.17.54.193:40503)
0812 03:52:28.133931 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.133934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.133972 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:52:28.134008 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.134009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.134017 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.134064 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:28.134312 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.135006 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.135958 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.136345 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.136442 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.136450 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.136622 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.136663 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:52:28.136765 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.133856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:54725 (local address 127.17.54.194:34551)
0812 03:52:28.133916 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.133929 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.134080 (+ 151us) server_negotiation.cc:407] Connection header received
0812 03:52:28.134181 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.134182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.134189 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.134231 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:28.134437 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.135070 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.135975 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.136340 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.136424 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.136433 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.136636 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.136695 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:52:28.136783 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.133747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:40503 (local address 127.17.54.195:60953)
0812 03:52:28.133857 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.133871 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.133922 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.134116 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.134117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.134119 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:28.134195 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.134197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.135094 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.135095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.135842 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.135844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.136441 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.136442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.136545 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.136548 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.136616 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.136674 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":2}
I20260812 03:52:28.136787 19496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.133755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:34551 (local address 127.17.54.195:54725)
0812 03:52:28.133999 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.134009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.134108 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.134279 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.134280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.134281 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:28.134363 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.134365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.135161 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.135163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.135881 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.135883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.136434 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.136435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.136549 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.136553 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.136616 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.136685 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":53,"thread_start_us":146,"threads_started":1}
I20260812 03:52:28.137184 19087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18face1d32e1480faea923f43361f8f8" candidate_uuid: "feda85fe092746c4a805c5b7921686f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b449d9c8ae694a3faba368b1d8777db8" is_pre_election: true
I20260812 03:52:28.137364 19087 raft_consensus.cc:2454] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate feda85fe092746c4a805c5b7921686f5 in term 0.
I20260812 03:52:28.137382 19243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18face1d32e1480faea923f43361f8f8" candidate_uuid: "feda85fe092746c4a805c5b7921686f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bb1f2101f254b94b88e969c1cb8047e" is_pre_election: true
I20260812 03:52:28.137552 19243 raft_consensus.cc:2454] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate feda85fe092746c4a805c5b7921686f5 in term 0.
I20260812 03:52:28.137629 19322 leader_election.cc:302] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b449d9c8ae694a3faba368b1d8777db8, feda85fe092746c4a805c5b7921686f5; no voters:
I20260812 03:52:28.137789 19492 raft_consensus.cc:2781] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:28.137872 19492 raft_consensus.cc:482] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:28.137904 19492 raft_consensus.cc:3037] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.138765 19492 raft_consensus.cc:504] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.138922 19492 leader_election.cc:288] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [CANDIDATE]: Term 1 election: Requested vote from peers b449d9c8ae694a3faba368b1d8777db8 (127.17.54.193:40503), 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551)
I20260812 03:52:28.139084 19243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18face1d32e1480faea923f43361f8f8" candidate_uuid: "feda85fe092746c4a805c5b7921686f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bb1f2101f254b94b88e969c1cb8047e"
I20260812 03:52:28.139108 19087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18face1d32e1480faea923f43361f8f8" candidate_uuid: "feda85fe092746c4a805c5b7921686f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b449d9c8ae694a3faba368b1d8777db8"
I20260812 03:52:28.139194 19243 raft_consensus.cc:3037] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.139191 19087 raft_consensus.cc:3037] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.139933 19087 raft_consensus.cc:2454] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feda85fe092746c4a805c5b7921686f5 in term 1.
I20260812 03:52:28.140017 19243 raft_consensus.cc:2454] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feda85fe092746c4a805c5b7921686f5 in term 1.
I20260812 03:52:28.140103 19322 leader_election.cc:302] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b449d9c8ae694a3faba368b1d8777db8, feda85fe092746c4a805c5b7921686f5; no voters:
I20260812 03:52:28.140198 19492 raft_consensus.cc:2781] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:28.140369 19492 raft_consensus.cc:686] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 1 LEADER]: Becoming Leader. State: Replica: feda85fe092746c4a805c5b7921686f5, State: Running, Role: LEADER
I20260812 03:52:28.140475 19492 consensus_queue.cc:237] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } }
I20260812 03:52:28.141399 18906 catalog_manager.cc:5495] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to feda85fe092746c4a805c5b7921686f5 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "feda85fe092746c4a805c5b7921686f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:28.153605 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.151125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.17.54.195:40507)
0812 03:52:28.151190 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.151192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.151205 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:52:28.151302 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.151303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.151320 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.151379 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:28.151529 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.152007 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.152832 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.153028 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.153082 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.153306 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:28.153399 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:28.153459 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.153510 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.153529 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:52:28.156111 19243 raft_consensus.cc:1261] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Refusing update from remote peer feda85fe092746c4a805c5b7921686f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:28.156504 19492 consensus_queue.cc:1035] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:28.158511 19087 raft_consensus.cc:1261] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Refusing update from remote peer feda85fe092746c4a805c5b7921686f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:28.158841 19497 consensus_queue.cc:1035] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:28.163507 18906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:28.163646 18906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:28.164561 18906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2: Sending DeleteTablet for 3 replicas of tablet 18face1d32e1480faea923f43361f8f8
I20260812 03:52:28.165071 19065 tablet_service.cc:1514] Processing DeleteTablet for tablet 18face1d32e1480faea923f43361f8f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:53452
I20260812 03:52:28.165246 19374 tablet_service.cc:1514] Processing DeleteTablet for tablet 18face1d32e1480faea923f43361f8f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:60574
I20260812 03:52:28.165444 19510 tablet_replica.cc:330] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: stopping tablet replica
I20260812 03:52:28.165529 19510 raft_consensus.cc:2229] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:28.165611 19510 raft_consensus.cc:2258] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:28.165727 19511 tablet_replica.cc:330] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: stopping tablet replica
I20260812 03:52:28.165792 19511 raft_consensus.cc:2229] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:28.165881 19511 raft_consensus.cc:2258] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:28.166289 19511 ts_tablet_manager.cc:1905] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:28.166986 19219 tablet_service.cc:1514] Processing DeleteTablet for tablet 18face1d32e1480faea923f43361f8f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:41026
I20260812 03:52:28.167706 18906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:28.167878 18906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:28.168385 19511 ts_tablet_manager.cc:1918] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:28.168452 19510 ts_tablet_manager.cc:1905] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:28.168579 19511 log.cc:1196] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/18face1d32e1480faea923f43361f8f8
I20260812 03:52:28.169020 19511 ts_tablet_manager.cc:1939] T 18face1d32e1480faea923f43361f8f8 P feda85fe092746c4a805c5b7921686f5: Deleting consensus metadata
I20260812 03:52:28.169250 19065 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd3fe84f89d468499f445951dbc03da (DEFAULT_TABLE table=test-table [id=1994dc86ab90415d8f62d9c8c34b6b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:28.169397 19065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd3fe84f89d468499f445951dbc03da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.169625 18894 catalog_manager.cc:4841] TS feda85fe092746c4a805c5b7921686f5 (127.17.54.195:40507): tablet 18face1d32e1480faea923f43361f8f8 (table test-table [id=c57dd9dbaf864e70abb5f4f533f04684]) successfully deleted
I20260812 03:52:28.169847 19219 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd3fe84f89d468499f445951dbc03da (DEFAULT_TABLE table=test-table [id=1994dc86ab90415d8f62d9c8c34b6b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:28.169992 19219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd3fe84f89d468499f445951dbc03da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.171140 19485 tablet_bootstrap.cc:492] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: Bootstrap starting.
I20260812 03:52:28.171185 19374 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd3fe84f89d468499f445951dbc03da (DEFAULT_TABLE table=test-table [id=1994dc86ab90415d8f62d9c8c34b6b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:28.171279 19374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd3fe84f89d468499f445951dbc03da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.171602 19485 tablet_bootstrap.cc:654] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.171962 19491 tablet_bootstrap.cc:492] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: Bootstrap starting.
I20260812 03:52:28.172394 19491 tablet_bootstrap.cc:654] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.172427 19485 tablet_bootstrap.cc:492] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: No bootstrap required, opened a new log
I20260812 03:52:28.172474 19485 ts_tablet_manager.cc:1397] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:28.172631 19484 tablet_bootstrap.cc:492] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: Bootstrap starting.
I20260812 03:52:28.172621 19485 raft_consensus.cc:348] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.172704 19485 raft_consensus.cc:374] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.172727 19485 raft_consensus.cc:729] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b449d9c8ae694a3faba368b1d8777db8, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.172782 19485 consensus_queue.cc:260] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.172881 19485 ts_tablet_manager.cc:1428] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:28.173062 19484 tablet_bootstrap.cc:654] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.173740 19484 tablet_bootstrap.cc:492] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: No bootstrap required, opened a new log
I20260812 03:52:28.173799 19484 ts_tablet_manager.cc:1397] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:28.173939 19484 raft_consensus.cc:348] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.174015 19484 raft_consensus.cc:374] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.174036 19484 raft_consensus.cc:729] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feda85fe092746c4a805c5b7921686f5, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.174055 19512 tablet_replica.cc:330] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: stopping tablet replica
I20260812 03:52:28.174080 19484 consensus_queue.cc:260] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.174170 19484 ts_tablet_manager.cc:1428] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:28.174252 19510 ts_tablet_manager.cc:1918] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:28.174345 19512 raft_consensus.cc:2229] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:28.174388 19510 log.cc:1196] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/18face1d32e1480faea923f43361f8f8
I20260812 03:52:28.174501 19512 raft_consensus.cc:2258] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:28.174935 19510 ts_tablet_manager.cc:1939] T 18face1d32e1480faea923f43361f8f8 P b449d9c8ae694a3faba368b1d8777db8: Deleting consensus metadata
I20260812 03:52:28.175179 19512 ts_tablet_manager.cc:1905] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:28.175485 18892 catalog_manager.cc:4841] TS b449d9c8ae694a3faba368b1d8777db8 (127.17.54.193:40503): tablet 18face1d32e1480faea923f43361f8f8 (table test-table [id=c57dd9dbaf864e70abb5f4f533f04684]) successfully deleted
I20260812 03:52:28.176775 19512 ts_tablet_manager.cc:1918] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:28.176864 19512 log.cc:1196] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/18face1d32e1480faea923f43361f8f8
I20260812 03:52:28.177141 19512 ts_tablet_manager.cc:1939] T 18face1d32e1480faea923f43361f8f8 P 4bb1f2101f254b94b88e969c1cb8047e: Deleting consensus metadata
I20260812 03:52:28.177569 18893 catalog_manager.cc:4841] TS 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551): tablet 18face1d32e1480faea923f43361f8f8 (table test-table [id=c57dd9dbaf864e70abb5f4f533f04684]) successfully deleted
I20260812 03:52:28.178759 19491 tablet_bootstrap.cc:492] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: No bootstrap required, opened a new log
I20260812 03:52:28.178834 19491 ts_tablet_manager.cc:1397] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260812 03:52:28.178994 19491 raft_consensus.cc:348] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.179066 19491 raft_consensus.cc:374] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.179081 19491 raft_consensus.cc:729] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb1f2101f254b94b88e969c1cb8047e, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.179143 19491 consensus_queue.cc:260] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.179248 19491 ts_tablet_manager.cc:1428] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260812 03:52:28.188665 19138 tablet.cc:2369] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:28.235826 19490 raft_consensus.cc:482] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:28.235936 19490 raft_consensus.cc:504] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.236320 19490 leader_election.cc:288] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551), feda85fe092746c4a805c5b7921686f5 (127.17.54.195:40507)
I20260812 03:52:28.240054 19518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.236509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:34551 (local address 127.17.54.193:51745)
0812 03:52:28.236687 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.236696 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.236751 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.236928 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.236929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.236931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:28.237017 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.237020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.238436 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.238437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.239186 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.239189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.239757 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.239759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.239845 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.239849 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.239919 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.239965 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":65,"threads_started":1}
I20260812 03:52:28.240042 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.236698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:51745 (local address 127.17.54.194:34551)
0812 03:52:28.236746 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.236749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.236760 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:28.236828 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.236829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.236840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.236909 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:28.237718 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.238262 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.239286 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.239664 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.239756 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.239766 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.239927 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.239966 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:28.240407 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.236583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:34937 (local address 127.17.54.195:40507)
0812 03:52:28.236637 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.236639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.236743 (+ 104us) server_negotiation.cc:407] Connection header received
0812 03:52:28.236848 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.236849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.236865 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.236914 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:28.237148 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.237938 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.239432 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.239993 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.240046 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.240054 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.240295 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.240338 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:52:28.240525 19243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd3fe84f89d468499f445951dbc03da" candidate_uuid: "b449d9c8ae694a3faba368b1d8777db8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bb1f2101f254b94b88e969c1cb8047e" is_pre_election: true
I20260812 03:52:28.240641 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.236491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:40507 (local address 127.17.54.193:34937)
0812 03:52:28.236673 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.236685 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.236767 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.236983 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.236983 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.236985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:28.237053 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.237054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.238072 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.238074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.239317 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.239321 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.240123 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.240124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.240202 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.240206 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.240511 (+ 305us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.240562 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260812 03:52:28.240646 19243 raft_consensus.cc:2454] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b449d9c8ae694a3faba368b1d8777db8 in term 0.
I20260812 03:52:28.240851 19010 leader_election.cc:302] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bb1f2101f254b94b88e969c1cb8047e, b449d9c8ae694a3faba368b1d8777db8; no voters:
I20260812 03:52:28.241098 19490 raft_consensus.cc:2781] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:28.241143 19490 raft_consensus.cc:482] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:28.241163 19490 raft_consensus.cc:3037] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.241212 19400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd3fe84f89d468499f445951dbc03da" candidate_uuid: "b449d9c8ae694a3faba368b1d8777db8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feda85fe092746c4a805c5b7921686f5" is_pre_election: true
I20260812 03:52:28.241329 19400 raft_consensus.cc:2454] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b449d9c8ae694a3faba368b1d8777db8 in term 0.
I20260812 03:52:28.242496 19490 raft_consensus.cc:504] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.242667 19490 leader_election.cc:288] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [CANDIDATE]: Term 1 election: Requested vote from peers 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551), feda85fe092746c4a805c5b7921686f5 (127.17.54.195:40507)
I20260812 03:52:28.243865 19243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd3fe84f89d468499f445951dbc03da" candidate_uuid: "b449d9c8ae694a3faba368b1d8777db8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bb1f2101f254b94b88e969c1cb8047e"
I20260812 03:52:28.243991 19243 raft_consensus.cc:3037] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.245040 19243 raft_consensus.cc:2454] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b449d9c8ae694a3faba368b1d8777db8 in term 1.
I20260812 03:52:28.245853 19010 leader_election.cc:302] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bb1f2101f254b94b88e969c1cb8047e, b449d9c8ae694a3faba368b1d8777db8; no voters:
I20260812 03:52:28.246919 19490 raft_consensus.cc:2781] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:28.247671 19400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd3fe84f89d468499f445951dbc03da" candidate_uuid: "b449d9c8ae694a3faba368b1d8777db8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feda85fe092746c4a805c5b7921686f5"
I20260812 03:52:28.247740 19490 raft_consensus.cc:686] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 1 LEADER]: Becoming Leader. State: Replica: b449d9c8ae694a3faba368b1d8777db8, State: Running, Role: LEADER
I20260812 03:52:28.247793 19400 raft_consensus.cc:3037] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.247826 19490 consensus_queue.cc:237] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.249380 19400 raft_consensus.cc:2454] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b449d9c8ae694a3faba368b1d8777db8 in term 1.
I20260812 03:52:28.250535 18908 catalog_manager.cc:5495] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b449d9c8ae694a3faba368b1d8777db8 (127.17.54.193). New cstate: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } health_report { overall_health: UNKNOWN } } }
W20260812 03:52:28.290812 19457 tablet.cc:2369] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:52:28 dist-test-slave-tm1g krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:28.356415 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.347713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41647 (local address 127.0.0.1:46170)
0812 03:52:28.347920 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.347934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.348014 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.348235 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.348236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.348573 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:28.348630 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.348632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.349375 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.349376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.350000 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.350001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.350614 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.350615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.350669 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.351387 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:28.351392 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:28.352842 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:28.355145 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:28.355153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:28.355155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:28.355366 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:28.355629 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:28.355630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:28.355630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:28.355700 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:28.356067 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:28.356075 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:28.356126 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.356209 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.356330 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":130,"threads_started":1}
I20260812 03:52:28.356587 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.347839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46170 (local address 127.17.54.254:41647)
0812 03:52:28.347916 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.347919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.348020 (+ 101us) server_negotiation.cc:407] Connection header received
0812 03:52:28.348075 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.348076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.348089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.348239 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:28.348724 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.349292 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.350094 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.350518 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.350614 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.352949 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:28.352951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:28.352952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:28.352962 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:28.355042 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.355452 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.355453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.355454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.355529 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.355779 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.355780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.355780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.355961 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:28.356043 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.356256 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.356413 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":41}
I20260812 03:52:28.359613 18908 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:46170:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:52:28.359997 18908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:28.361703 19374 tablet_service.cc:1467] Processing CreateTablet for tablet 102909936b6d49bca18771802b958b51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=79e941bd34094e9ca8b19e1816f4c4c6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:28.361871 19065 tablet_service.cc:1467] Processing CreateTablet for tablet 102909936b6d49bca18771802b958b51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=79e941bd34094e9ca8b19e1816f4c4c6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:28.361945 19374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102909936b6d49bca18771802b958b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.362062 19065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102909936b6d49bca18771802b958b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.363555 19484 tablet_bootstrap.cc:492] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: Bootstrap starting.
I20260812 03:52:28.363955 19484 tablet_bootstrap.cc:654] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.364068 19485 tablet_bootstrap.cc:492] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: Bootstrap starting.
I20260812 03:52:28.364176 19219 tablet_service.cc:1467] Processing CreateTablet for tablet 102909936b6d49bca18771802b958b51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=79e941bd34094e9ca8b19e1816f4c4c6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:28.364355 19219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102909936b6d49bca18771802b958b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:28.364490 19485 tablet_bootstrap.cc:654] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.365123 19485 tablet_bootstrap.cc:492] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: No bootstrap required, opened a new log
I20260812 03:52:28.365159 19485 ts_tablet_manager.cc:1397] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:28.365278 19485 raft_consensus.cc:348] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.365345 19485 raft_consensus.cc:374] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.365357 19485 raft_consensus.cc:729] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b449d9c8ae694a3faba368b1d8777db8, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.365420 19485 consensus_queue.cc:260] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.365535 19485 ts_tablet_manager.cc:1428] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:28.365607 19519 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.365700 19519 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:28.366407 19491 tablet_bootstrap.cc:492] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: Bootstrap starting.
I20260812 03:52:28.366753 19491 tablet_bootstrap.cc:654] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:28.367492 19484 tablet_bootstrap.cc:492] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: No bootstrap required, opened a new log
I20260812 03:52:28.367611 19484 ts_tablet_manager.cc:1397] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260812 03:52:28.367682 19491 tablet_bootstrap.cc:492] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: No bootstrap required, opened a new log
I20260812 03:52:28.367722 19491 ts_tablet_manager.cc:1397] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:28.367739 19484 raft_consensus.cc:348] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.367816 19484 raft_consensus.cc:374] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.367835 19484 raft_consensus.cc:729] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feda85fe092746c4a805c5b7921686f5, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.367859 19491 raft_consensus.cc:348] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.367892 19484 consensus_queue.cc:260] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.367924 19491 raft_consensus.cc:374] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:28.367946 19491 raft_consensus.cc:729] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb1f2101f254b94b88e969c1cb8047e, State: Initialized, Role: FOLLOWER
I20260812 03:52:28.367990 19484 ts_tablet_manager.cc:1428] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:28.368032 19491 consensus_queue.cc:260] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.368033 19492 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.368139 19491 ts_tablet_manager.cc:1428] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:28.368168 19492 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:28.368314 19495 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.368430 19495 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:28.403947 19519 raft_consensus.cc:482] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:28.404086 19519 raft_consensus.cc:504] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.404331 19519 leader_election.cc:288] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551), feda85fe092746c4a805c5b7921686f5 (127.17.54.195:40507)
I20260812 03:52:28.404577 19243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102909936b6d49bca18771802b958b51" candidate_uuid: "b449d9c8ae694a3faba368b1d8777db8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bb1f2101f254b94b88e969c1cb8047e" is_pre_election: true
I20260812 03:52:28.404577 19400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102909936b6d49bca18771802b958b51" candidate_uuid: "b449d9c8ae694a3faba368b1d8777db8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feda85fe092746c4a805c5b7921686f5" is_pre_election: true
I20260812 03:52:28.404683 19243 raft_consensus.cc:2454] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b449d9c8ae694a3faba368b1d8777db8 in term 0.
I20260812 03:52:28.404683 19400 raft_consensus.cc:2454] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b449d9c8ae694a3faba368b1d8777db8 in term 0.
I20260812 03:52:28.404843 19010 leader_election.cc:302] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bb1f2101f254b94b88e969c1cb8047e, b449d9c8ae694a3faba368b1d8777db8; no voters:
I20260812 03:52:28.405012 19519 raft_consensus.cc:2781] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:28.405061 19519 raft_consensus.cc:482] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:28.405076 19519 raft_consensus.cc:3037] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.405872 19519 raft_consensus.cc:504] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.406018 19519 leader_election.cc:288] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [CANDIDATE]: Term 1 election: Requested vote from peers 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551), feda85fe092746c4a805c5b7921686f5 (127.17.54.195:40507)
I20260812 03:52:28.406212 19400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102909936b6d49bca18771802b958b51" candidate_uuid: "b449d9c8ae694a3faba368b1d8777db8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feda85fe092746c4a805c5b7921686f5"
I20260812 03:52:28.406288 19400 raft_consensus.cc:3037] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.406288 19243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102909936b6d49bca18771802b958b51" candidate_uuid: "b449d9c8ae694a3faba368b1d8777db8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bb1f2101f254b94b88e969c1cb8047e"
I20260812 03:52:28.406419 19243 raft_consensus.cc:3037] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:28.406968 19400 raft_consensus.cc:2454] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b449d9c8ae694a3faba368b1d8777db8 in term 1.
I20260812 03:52:28.407161 19011 leader_election.cc:302] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b449d9c8ae694a3faba368b1d8777db8, feda85fe092746c4a805c5b7921686f5; no voters:
I20260812 03:52:28.407287 19519 raft_consensus.cc:2781] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:28.407337 19519 raft_consensus.cc:686] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [term 1 LEADER]: Becoming Leader. State: Replica: b449d9c8ae694a3faba368b1d8777db8, State: Running, Role: LEADER
I20260812 03:52:28.407403 19519 consensus_queue.cc:237] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } }
I20260812 03:52:28.407503 19243 raft_consensus.cc:2454] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b449d9c8ae694a3faba368b1d8777db8 in term 1.
I20260812 03:52:28.407773 19490 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: TxnStatusTablet state changed. Reason: New leader b449d9c8ae694a3faba368b1d8777db8. Latest consensus state: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.407886 19490 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:28.408097 19534 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:52:28.408249 19534 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:52:28.408610 18906 catalog_manager.cc:5495] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b449d9c8ae694a3faba368b1d8777db8 (127.17.54.193). New cstate: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:28.416716 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.412681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:40503 (local address 127.0.0.1:37850)
0812 03:52:28.412773 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.412788 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.412852 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.413219 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.413220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.413221 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:28.413279 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.413282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.414335 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.414337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.415392 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.415396 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.416016 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.416017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.416092 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.416116 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:28.416488 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:28.416501 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.416552 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.416610 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:52:28.416744 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.412959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.17.54.193:40503)
0812 03:52:28.413040 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.413042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.413053 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:28.413091 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.413092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.413114 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.413207 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:28.413359 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.414212 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.415512 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.415913 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.416008 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.416233 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:28.416399 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:28.416489 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.416569 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.416602 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:52:28.418514 19243 raft_consensus.cc:1261] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Refusing update from remote peer b449d9c8ae694a3faba368b1d8777db8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:28.418536 19400 raft_consensus.cc:1261] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Refusing update from remote peer b449d9c8ae694a3faba368b1d8777db8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:28.418820 19490 consensus_queue.cc:1035] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:28.418951 19519 consensus_queue.cc:1035] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:28.419620 19495 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: TxnStatusTablet state changed. Reason: New leader b449d9c8ae694a3faba368b1d8777db8. Latest consensus state: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.419728 19495 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:28.420246 19492 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: TxnStatusTablet state changed. Reason: New leader b449d9c8ae694a3faba368b1d8777db8. Latest consensus state: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.420347 19492 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:28.420913 19490 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.421015 19490 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:28.421262 19519 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.421347 19519 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P b449d9c8ae694a3faba368b1d8777db8: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:28.422159 19501 mvcc.cc:205] Tried to move back new op lower bound from 7317531641519702016 to 7317531641477107712. Current Snapshot: MvccSnapshot[applied={T|T < 7317531641519702016 or (T in {7317531641519702016})}]
I20260812 03:52:28.422276 19495 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.422391 19495 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P 4bb1f2101f254b94b88e969c1cb8047e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:28.422734 19492 tablet_replica.cc:439] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b449d9c8ae694a3faba368b1d8777db8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b449d9c8ae694a3faba368b1d8777db8" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40503 } } peers { permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 } } peers { permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 } } }
I20260812 03:52:28.422827 19492 tablet_replica.cc:442] T 102909936b6d49bca18771802b958b51 P feda85fe092746c4a805c5b7921686f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:28.426117 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.423781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.17.54.193:40503)
0812 03:52:28.423937 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.423939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.423945 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:52:28.423962 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.423963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.423972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.424050 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:28.424183 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.424665 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.425421 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.425632 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.425711 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.425867 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:28.425903 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:28.425954 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.426008 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.426037 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260812 03:52:28.432116 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.429067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:34991 (local address 127.17.54.193:40503)
0812 03:52:28.429111 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.429114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.429290 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:52:28.429337 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.429338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.429346 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.429393 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:28.429535 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.429971 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.431145 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.431325 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.431409 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.431793 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:28.431831 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:28.431912 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.431999 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.432027 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":22}
I20260812 03:52:28.432190 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.429019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:40503 (local address 127.17.54.193:34991)
0812 03:52:28.429207 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:28.429219 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:28.429284 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:28.429397 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:28.429398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:28.429399 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:28.429478 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.429479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.430038 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.430039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.431047 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:28.431050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.431407 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.431408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:28.431489 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.431505 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:28.431915 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:28.431917 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:52:28.431994 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:52:28.432048 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260812 03:52:28.434877 19400 raft_consensus.cc:1261] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Refusing update from remote peer b449d9c8ae694a3faba368b1d8777db8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:28.434883 19243 raft_consensus.cc:1261] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Refusing update from remote peer b449d9c8ae694a3faba368b1d8777db8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:28.435102 19545 consensus_queue.cc:1035] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feda85fe092746c4a805c5b7921686f5" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:28.435206 19490 consensus_queue.cc:1035] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bb1f2101f254b94b88e969c1cb8047e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:28.436612 19506 mvcc.cc:205] Tried to move back new op lower bound from 7317531641587421184 to 7317531640823844864. Current Snapshot: MvccSnapshot[applied={T|T < 7317531641587421184}]
I20260812 03:52:28.436836 19500 mvcc.cc:205] Tried to move back new op lower bound from 7317531641587421184 to 7317531640823844864. Current Snapshot: MvccSnapshot[applied={T|T < 7317531641587421184}]
I20260812 03:52:28.450224 18908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:28.450389 18908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:28.451186 18908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43031759fedd45e690261fa33331a9e2: Sending DeleteTablet for 3 replicas of tablet 1dd3fe84f89d468499f445951dbc03da
I20260812 03:52:28.451524 19374 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dd3fe84f89d468499f445951dbc03da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:60574
I20260812 03:52:28.451562 19219 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dd3fe84f89d468499f445951dbc03da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:41026
I20260812 03:52:28.451649 19511 tablet_replica.cc:330] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: stopping tablet replica
I20260812 03:52:28.451666 19512 tablet_replica.cc:330] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: stopping tablet replica
I20260812 03:52:28.451725 19511 raft_consensus.cc:2229] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:28.451736 19512 raft_consensus.cc:2229] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:28.451771 19511 raft_consensus.cc:2258] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:28.451774 19512 raft_consensus.cc:2258] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:28.452195 19512 ts_tablet_manager.cc:1905] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:28.452313 19065 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dd3fe84f89d468499f445951dbc03da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:53452
I20260812 03:52:28.452430 19510 tablet_replica.cc:330] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: stopping tablet replica
I20260812 03:52:28.452495 19510 raft_consensus.cc:2229] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:28.452576 19510 raft_consensus.cc:2258] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:28.452874 19510 ts_tablet_manager.cc:1905] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:28.452970 19511 ts_tablet_manager.cc:1905] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:28.454275 19512 ts_tablet_manager.cc:1918] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:28.454370 19512 log.cc:1196] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/1dd3fe84f89d468499f445951dbc03da
I20260812 03:52:28.454639 19512 ts_tablet_manager.cc:1939] T 1dd3fe84f89d468499f445951dbc03da P 4bb1f2101f254b94b88e969c1cb8047e: Deleting consensus metadata
I20260812 03:52:28.454627 19510 ts_tablet_manager.cc:1918] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:28.454686 19510 log.cc:1196] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/1dd3fe84f89d468499f445951dbc03da
I20260812 03:52:28.454666 19511 ts_tablet_manager.cc:1918] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:28.454730 19511 log.cc:1196] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/1dd3fe84f89d468499f445951dbc03da
I20260812 03:52:28.454917 19510 ts_tablet_manager.cc:1939] T 1dd3fe84f89d468499f445951dbc03da P b449d9c8ae694a3faba368b1d8777db8: Deleting consensus metadata
I20260812 03:52:28.454964 19511 ts_tablet_manager.cc:1939] T 1dd3fe84f89d468499f445951dbc03da P feda85fe092746c4a805c5b7921686f5: Deleting consensus metadata
I20260812 03:52:28.454988 18893 catalog_manager.cc:4841] TS 4bb1f2101f254b94b88e969c1cb8047e (127.17.54.194:34551): tablet 1dd3fe84f89d468499f445951dbc03da (table test-table [id=1994dc86ab90415d8f62d9c8c34b6b3f]) successfully deleted
I20260812 03:52:28.455505 18894 catalog_manager.cc:4841] TS feda85fe092746c4a805c5b7921686f5 (127.17.54.195:40507): tablet 1dd3fe84f89d468499f445951dbc03da (table test-table [id=1994dc86ab90415d8f62d9c8c34b6b3f]) successfully deleted
I20260812 03:52:28.455506 18892 catalog_manager.cc:4841] TS b449d9c8ae694a3faba368b1d8777db8 (127.17.54.193:40503): tablet 1dd3fe84f89d468499f445951dbc03da (table test-table [id=1994dc86ab90415d8f62d9c8c34b6b3f]) successfully deleted
Aug 12 03:52:28 dist-test-slave-tm1g krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506748, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:28.468457 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.452570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37874 (local address 127.17.54.193:40503)
0812 03:52:28.452609 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.452611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.458549 (+ 5938us) server_negotiation.cc:407] Connection header received
0812 03:52:28.458581 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.458582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.458593 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.458667 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:28.459219 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.459995 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.460850 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.461161 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.461232 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.464783 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:28.464796 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:28.464796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:28.464807 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:28.466944 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.467366 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.467371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.467372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.467445 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.467707 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.467708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.467708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.467920 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:28.468152 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.468251 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.468344 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 03:52:28.468827 19130 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37874
I20260812 03:52:28.476749 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:28.469979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.17.54.254:41647)
0812 03:52:28.470032 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:28.470034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:28.470112 (+ 78us) server_negotiation.cc:407] Connection header received
0812 03:52:28.470176 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:28.470177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:28.470187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:28.470235 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:28.470917 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.471530 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.472546 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:28.472736 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:28.472792 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:28.474165 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:28.474167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:28.474167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:28.474174 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:28.475642 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.476054 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.476055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.476056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.476111 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:28.476315 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:28.476315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:28.476316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:28.476417 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:28.476492 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:28.476621 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 03:52:28.476681 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 03:52:28.477013 18908 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40094
I20260812 03:52:28.477849 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 18988
I20260812 03:52:28.487217 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 19147
I20260812 03:52:28.497236 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 19302
I20260812 03:52:28.507508 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 18872
2026-08-12T03:52:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2436 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:28 dist-test-slave-tm1g krb5kdc[19559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:28 dist-test-slave-tm1g krb5kdc[19559](info): set up 2 sockets
Aug 12 03:52:28 dist-test-slave-tm1g krb5kdc[19559](info): commencing operation
krb5kdc: starting...
W20260812 03:52:29.669907 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:29 dist-test-slave-tm1g krb5kdc[19559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:29.798524 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:46381
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:46381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:29.881999 19574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:29.882251 19574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:29.882289 19574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:29.883481 19574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:29.883558 19574 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:52:29.883579 19574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:29.883601 19574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:29.883632 19574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:29.883654 19574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:29.885224 19574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:46381
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:46381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:29.885538 19574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:29.885794 19574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:29.890913 19579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:29.890938 19582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:29.890978 19580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:29.891386 19574 server_base.cc:1034] running on GCE node
I20260812 03:52:29.891733 19574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:29.892109 19574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:29.893316 19574 hybrid_clock.cc:648] HybridClock initialized: now 1786506749893292 us; error 55 us; skew 500 ppm
Aug 12 03:52:29 dist-test-slave-tm1g krb5kdc[19559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506749, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:29.899924 19574 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:29.900514 19574 webserver.cc:460] Webserver started at http://127.17.54.254:43291/ using document root <none> and password file <none>
I20260812 03:52:29.900786 19574 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:29.900856 19574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:29.901011 19574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:29.902145 19574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "371a132e40be4183931295c089766fd0"
format_stamp: "Formatted at 2026-08-12 03:52:29 on dist-test-slave-tm1g"
server_key: "12c9966303ca22b5a9d301d4e50e50fd"
server_key_iv: "47bdd7b5123724e247d213f68ec6cfb6"
server_key_version: "encryptionkey@0"
I20260812 03:52:29.902544 19574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "371a132e40be4183931295c089766fd0"
format_stamp: "Formatted at 2026-08-12 03:52:29 on dist-test-slave-tm1g"
server_key: "12c9966303ca22b5a9d301d4e50e50fd"
server_key_iv: "47bdd7b5123724e247d213f68ec6cfb6"
server_key_version: "encryptionkey@0"
I20260812 03:52:29.904002 19574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:29.904942 19589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:29.905258 19574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:29.905424 19574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "371a132e40be4183931295c089766fd0"
format_stamp: "Formatted at 2026-08-12 03:52:29 on dist-test-slave-tm1g"
server_key: "12c9966303ca22b5a9d301d4e50e50fd"
server_key_iv: "47bdd7b5123724e247d213f68ec6cfb6"
server_key_version: "encryptionkey@0"
I20260812 03:52:29.905511 19574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:29.925339 19574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:29.928153 19574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:29.928354 19574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:29.933981 19574 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:46381
I20260812 03:52:29.934047 19656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:46381 every 8 connection(s)
I20260812 03:52:29.934490 19574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:29.935176 19657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:29.937542 19657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: Bootstrap starting.
I20260812 03:52:29.938166 19657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:29.938506 19657 log.cc:824] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:29.939338 19657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: No bootstrap required, opened a new log
I20260812 03:52:29.941668 19657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371a132e40be4183931295c089766fd0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 46381 } }
I20260812 03:52:29.941813 19657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:29.941835 19657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 371a132e40be4183931295c089766fd0, State: Initialized, Role: FOLLOWER
I20260812 03:52:29.942001 19657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371a132e40be4183931295c089766fd0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 46381 } }
I20260812 03:52:29.942075 19657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:29.942096 19657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:29.942165 19657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:29.942813 19657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371a132e40be4183931295c089766fd0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 46381 } }
I20260812 03:52:29.942904 19657 leader_election.cc:302] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 371a132e40be4183931295c089766fd0; no voters:
I20260812 03:52:29.943125 19657 leader_election.cc:288] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:29.943157 19661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:29.943409 19661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [term 1 LEADER]: Becoming Leader. State: Replica: 371a132e40be4183931295c089766fd0, State: Running, Role: LEADER
I20260812 03:52:29.943427 19657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:29.943528 19661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371a132e40be4183931295c089766fd0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 46381 } }
I20260812 03:52:29.943936 19663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 371a132e40be4183931295c089766fd0. Latest consensus state: current_term: 1 leader_uuid: "371a132e40be4183931295c089766fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371a132e40be4183931295c089766fd0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 46381 } } }
I20260812 03:52:29.943936 19662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "371a132e40be4183931295c089766fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371a132e40be4183931295c089766fd0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 46381 } } }
I20260812 03:52:29.944057 19663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:29.944065 19662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:29.944324 19667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:29.944545 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 19574
I20260812 03:52:29.944649 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:29.944849 17627 external_mini_cluster.cc:1428] Setting key 38e3bc4929e0089f83f92bfecf247ad7
I20260812 03:52:29.945205 19667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:29.947203 19667 catalog_manager.cc:1347] Generated new cluster ID: 3f2f52ca28da45ccba99f4c17570c830
I20260812 03:52:29.947257 19667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:52:29 dist-test-slave-tm1g krb5kdc[19559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:29.954550 19667 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:29.955061 19667 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:29.957417 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:29.945421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.17.54.254:46381)
0812 03:52:29.945734 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:29.945743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:29.945775 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:52:29.946045 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:29.946054 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:29.946118 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:29.946281 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:29.947042 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:29.948464 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:29.949597 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:29.950226 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:29.950359 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:29.953345 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:29.953350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:29.953350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:29.953363 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:29.955506 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:29.956064 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:29.956068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:29.956068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:29.956188 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:29.956449 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:29.956450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:29.956450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:29.956754 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:29.956918 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:29.957021 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 03:52:29.957180 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":92,"threads_started":1}
I20260812 03:52:29.963058 19667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: Generated new TSK 0
I20260812 03:52:29.963243 19667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:29.997272 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:46381
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:30.085880 19686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:30.086119 19686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:30.086148 19686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:30.087404 19686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:30.087464 19686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:30.087486 19686 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:52:30.087499 19686 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:30.087517 19686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:30.089022 19686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:46381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:30.089254 19686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:30.089460 19686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:30.095217 19692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:30.095182 19693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:30.095433 19686 server_base.cc:1034] running on GCE node
W20260812 03:52:30.095508 19695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:30.095809 19686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:30.096179 19686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:30.097438 19686 hybrid_clock.cc:648] HybridClock initialized: now 1786506750097413 us; error 46 us; skew 500 ppm
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506750, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:30.100243 19686 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:30.100754 19686 webserver.cc:460] Webserver started at http://127.17.54.193:34769/ using document root <none> and password file <none>
I20260812 03:52:30.101024 19686 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:30.101073 19686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:30.101227 19686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:30.102442 19686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "04ad518a9ffb4452b0a9785b4e8ae419"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "d0f7a06de766c27eb605d8801208bdbf"
server_key_iv: "7680c13674269f965673447b6b34cb51"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.102851 19686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "04ad518a9ffb4452b0a9785b4e8ae419"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "d0f7a06de766c27eb605d8801208bdbf"
server_key_iv: "7680c13674269f965673447b6b34cb51"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.104413 19686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:52:30.105371 19704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.105588 19686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:30.105693 19686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "04ad518a9ffb4452b0a9785b4e8ae419"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "d0f7a06de766c27eb605d8801208bdbf"
server_key_iv: "7680c13674269f965673447b6b34cb51"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.105754 19686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:30.125052 19686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:30.127421 19686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:30.127555 19686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:30.128082 19686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:30.128126 19686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.128150 19686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:30.128163 19686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.135452 19686 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:33099
I20260812 03:52:30.135550 19836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:33099 every 8 connection(s)
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506750, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:30.135888 19686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:30.139304 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.128745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:52549 (local address 127.17.54.254:46381)
0812 03:52:30.128820 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.128822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.129074 (+ 252us) server_negotiation.cc:407] Connection header received
0812 03:52:30.129754 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.129756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.129771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.129855 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:30.130682 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.131574 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.133254 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.133597 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.133686 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.136394 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:30.136396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:30.136396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:30.136404 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:30.137896 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.138355 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.138356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.138357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.138455 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.138700 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.138701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.138701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.138819 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:30.138873 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.139144 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.139231 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:52:30.139444 19716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.128706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:46381 (local address 127.17.54.193:52549)
0812 03:52:30.128936 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.128977 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.129630 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.129935 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.129941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.130327 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:30.130599 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.130604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.131699 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.131700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.133154 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.133157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.133715 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.133716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.133831 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.134637 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:30.134647 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:30.136265 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:30.137985 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.137991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.137992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.138216 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.138530 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.138531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.138531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.138638 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.138902 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:30.138907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:30.139024 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.139113 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.139237 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":157,"threads_started":1}
I20260812 03:52:30.140002 19837 heartbeater.cc:347] Connected to a master server at 127.17.54.254:46381
I20260812 03:52:30.140082 19837 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:30.140272 19837 heartbeater.cc:511] Master 127.17.54.254:46381 requested a full tablet report, sending...
I20260812 03:52:30.140830 19607 ts_manager.cc:194] Registered new tserver with Master: 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099)
I20260812 03:52:30.141582 19607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:52549
I20260812 03:52:30.143466 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 19686
I20260812 03:52:30.143575 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:30.143787 17627 external_mini_cluster.cc:1428] Setting key fadd8a47cd4ce8549c2ff2aa38229795
I20260812 03:52:30.147109 19844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.141119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:46381 (local address 127.17.54.193:58493)
0812 03:52:30.141246 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.141258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.141318 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.141471 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.141472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.141667 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:30.141739 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.141741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.142401 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.142402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.142941 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.142943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.143356 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.143357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.143430 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.143928 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:30.143933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:30.144332 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:30.146106 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.146107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.146107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.146267 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.146532 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.146532 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.146533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.146589 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.146854 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:30.146855 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:30.146908 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.146960 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.147037 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260812 03:52:30.147219 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.141194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:58493 (local address 127.17.54.254:46381)
0812 03:52:30.141237 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.141239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.141301 (+ 62us) server_negotiation.cc:407] Connection header received
0812 03:52:30.141375 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.141376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.141385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.141464 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:30.141805 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.142276 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.143026 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.143279 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.143353 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.144405 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:30.144406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:30.144407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:30.144413 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:30.146016 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.146380 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.146381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.146382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.146460 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.146640 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.146641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.146641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.146788 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:30.146861 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.147021 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.147107 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:30.174772 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:46381
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:30.262773 19847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:30.262974 19847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:30.262998 19847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:30.264158 19847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:30.264204 19847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:30.264225 19847 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:52:30.264237 19847 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:30.264254 19847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:30.265758 19847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:46381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:30.266002 19847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:30.266247 19847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:30.271294 19853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:30.271340 19854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:30.271341 19856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:30.271910 19847 server_base.cc:1034] running on GCE node
I20260812 03:52:30.272146 19847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:30.272409 19847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:30.273591 19847 hybrid_clock.cc:648] HybridClock initialized: now 1786506750273581 us; error 40 us; skew 500 ppm
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506750, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:30.276218 19847 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:30.276697 19847 webserver.cc:460] Webserver started at http://127.17.54.194:34261/ using document root <none> and password file <none>
I20260812 03:52:30.276937 19847 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:30.276985 19847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:30.277122 19847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:30.278191 19847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "5beed0f7b84d40da9270b2cea8842804"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "24985af95ffa3a9a01b7f307bb74b18b"
server_key_iv: "3e3cee64bf02a7844f3f160a2bc24cac"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.278635 19847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "5beed0f7b84d40da9270b2cea8842804"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "24985af95ffa3a9a01b7f307bb74b18b"
server_key_iv: "3e3cee64bf02a7844f3f160a2bc24cac"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.280144 19847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:52:30.281116 19865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.281378 19847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:30.281510 19847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "5beed0f7b84d40da9270b2cea8842804"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "24985af95ffa3a9a01b7f307bb74b18b"
server_key_iv: "3e3cee64bf02a7844f3f160a2bc24cac"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.281606 19847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:30.298959 19847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:30.301946 19847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:30.302103 19847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:30.302721 19847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:30.302763 19847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.302793 19847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:30.302850 19847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.310341 19847 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:38037
I20260812 03:52:30.310431 19990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:38037 every 8 connection(s)
I20260812 03:52:30.310797 19847 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506750, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:30.314196 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.303323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:50039 (local address 127.17.54.254:46381)
0812 03:52:30.303417 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.303419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.303625 (+ 206us) server_negotiation.cc:407] Connection header received
0812 03:52:30.304189 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.304190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.304203 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.304280 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:30.305366 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.306208 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.308027 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.308383 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.308480 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.311495 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:30.311496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:30.311496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:30.311503 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:30.312768 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.313205 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.313206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.313206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.313274 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.313592 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.313593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.313593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.313721 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:30.313790 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.314056 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.314129 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:52:30.314517 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.303295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:46381 (local address 127.17.54.194:50039)
0812 03:52:30.303491 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.303549 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.304090 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.304420 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.304427 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.304872 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:30.305203 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.305209 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.306360 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.306362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.307912 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.307916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.308524 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.308526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.308803 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.309470 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:30.309481 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:30.311399 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:30.312859 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.312867 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.312869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.313109 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.313392 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.313393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.313394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.313519 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.313797 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:30.313803 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:30.313930 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.314028 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.314161 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":123,"threads_started":1}
I20260812 03:52:30.315099 19991 heartbeater.cc:347] Connected to a master server at 127.17.54.254:46381
I20260812 03:52:30.315220 19991 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:30.315461 19991 heartbeater.cc:511] Master 127.17.54.254:46381 requested a full tablet report, sending...
I20260812 03:52:30.315888 19607 ts_manager.cc:194] Registered new tserver with Master: 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037)
I20260812 03:52:30.316401 19607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:50039
I20260812 03:52:30.320910 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 19847
I20260812 03:52:30.320994 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:30.321226 17627 external_mini_cluster.cc:1428] Setting key 0eb270d375d010b02b9dd92d915e9ba1
I20260812 03:52:30.322135 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.316262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:33921 (local address 127.17.54.254:46381)
0812 03:52:30.316298 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.316299 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.316437 (+ 138us) server_negotiation.cc:407] Connection header received
0812 03:52:30.316461 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.316462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.316473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.316537 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:30.316849 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.317332 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.318046 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.318257 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.318371 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.319339 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:30.319341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:30.319341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:30.319349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:30.320697 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.321178 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.321180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.321180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.321242 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.321510 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.321511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.321511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.321710 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:30.321781 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.321949 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.322037 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:52:30.322151 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.316192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:46381 (local address 127.17.54.194:33921)
0812 03:52:30.316327 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.316338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.316427 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.316554 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.316556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.316711 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:30.316770 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.316771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.317433 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.317434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.317962 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.317964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.318380 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.318382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.318463 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.318922 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:30.318927 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:30.319218 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:30.320843 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.320844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.320845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.321083 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.321340 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.321341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.321341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.321433 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.321783 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:30.321784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:30.321834 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.321923 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.322042 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:30.354010 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:46381
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:30.443331 20000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:30.443617 20000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:30.443681 20000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:30.445101 20000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:30.445142 20000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:30.445163 20000 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:52:30.445175 20000 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:30.445194 20000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:30.446789 20000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:46381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:30.446985 20000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:30.447165 20000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:30.452410 20008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:30.452464 20010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:30.452564 20013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:30.452762 20000 server_base.cc:1034] running on GCE node
I20260812 03:52:30.452970 20000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:30.453181 20000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:30.454357 20000 hybrid_clock.cc:648] HybridClock initialized: now 1786506750454338 us; error 35 us; skew 500 ppm
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506750, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:30.457007 20000 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:30.457581 20000 webserver.cc:460] Webserver started at http://127.17.54.195:40233/ using document root <none> and password file <none>
I20260812 03:52:30.457865 20000 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:30.457949 20000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:30.458117 20000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:30.459355 20000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "f2bc4088aad94fa3b331075ab65e6b61"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "3feb30ada560ea5256269eb183f20f85"
server_key_iv: "10163b66a2a678b9d488b1ff29499038"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.459765 20000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "f2bc4088aad94fa3b331075ab65e6b61"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "3feb30ada560ea5256269eb183f20f85"
server_key_iv: "10163b66a2a678b9d488b1ff29499038"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.461248 20000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:30.462177 20020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.462430 20000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:30.462540 20000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "f2bc4088aad94fa3b331075ab65e6b61"
format_stamp: "Formatted at 2026-08-12 03:52:30 on dist-test-slave-tm1g"
server_key: "3feb30ada560ea5256269eb183f20f85"
server_key_iv: "10163b66a2a678b9d488b1ff29499038"
server_key_version: "encryptionkey@0"
I20260812 03:52:30.462627 20000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:30.477146 20000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:30.479590 20000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:30.479790 20000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:30.480477 20000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:30.480535 20000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.480568 20000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:30.480587 20000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506750, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:30.488440 20000 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:45881
I20260812 03:52:30.488497 20145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:45881 every 8 connection(s)
I20260812 03:52:30.488863 20000 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:30.490453 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 20000
I20260812 03:52:30.490548 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:30.490736 17627 external_mini_cluster.cc:1428] Setting key 15c11a878f4ac0787c0cb49ba9d825af
I20260812 03:52:30.491571 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.480923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:46381 (local address 127.17.54.195:44215)
0812 03:52:30.481106 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.481154 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.481722 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.482043 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.482048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.482425 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:30.482644 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.482650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.483686 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.483687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.484840 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.484843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.485335 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.485336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.485502 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.486121 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:30.486132 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:30.488213 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:30.490030 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.490035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.490037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.490252 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.490526 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.490527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.490527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.490663 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.490975 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:30.490982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:30.491126 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.491232 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.491408 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260812 03:52:30.491824 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.480967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:44215 (local address 127.17.54.254:46381)
0812 03:52:30.481045 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.481048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.481259 (+ 211us) server_negotiation.cc:407] Connection header received
0812 03:52:30.481834 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.481835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.481852 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.481943 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:30.482733 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.483587 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.484958 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.485196 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.485331 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.488312 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:30.488314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:30.488315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:30.488324 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:30.489954 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.490375 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.490376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.490377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.490444 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.490752 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.490753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.490753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.490902 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:30.490986 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.491697 (+ 711us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.491767 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:52:30.492195 20146 heartbeater.cc:347] Connected to a master server at 127.17.54.254:46381
I20260812 03:52:30.492311 20146 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:30.492493 20146 heartbeater.cc:511] Master 127.17.54.254:46381 requested a full tablet report, sending...
I20260812 03:52:30.492897 19609 ts_manager.cc:194] Registered new tserver with Master: f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:30.493356 19609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:44215
I20260812 03:52:30.499024 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.493263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:52009 (local address 127.17.54.254:46381)
0812 03:52:30.493304 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.493306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.493436 (+ 130us) server_negotiation.cc:407] Connection header received
0812 03:52:30.493508 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.493508 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.493518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.493566 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:30.493888 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.494438 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.495188 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.495398 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.495483 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.496428 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:30.496429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:30.496430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:30.496437 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:30.497819 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.498194 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.498195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.498196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.498253 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.498498 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.498499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.498499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.498617 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:30.498693 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.498850 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.498927 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:52:30.499038 20152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.493205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:46381 (local address 127.17.54.195:52009)
0812 03:52:30.493358 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.493375 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.493449 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.493568 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.493569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.493746 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:30.493804 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.493806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.494527 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.494528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.495099 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.495101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.495486 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.495487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.495563 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.496024 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:30.496029 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:30.496311 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:30.497931 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.497932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.497932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.498118 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.498365 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.498366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.498366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.498432 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.498685 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:30.498686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:30.498735 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.498823 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.498936 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260812 03:52:30.502519 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:30.527271 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.519899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46224 (local address 127.17.54.254:46381)
0812 03:52:30.519974 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.519976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.519990 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:52:30.520056 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.520057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.520067 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.520118 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:30.520683 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.521186 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.521967 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.522159 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.522210 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.524567 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:30.524568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:30.524569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:30.524576 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:30.525985 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.526463 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.526464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.526465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.526534 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.526741 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.526742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.526742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.526860 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:30.526967 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.527081 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.527163 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:52:30.528952 19609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:30.530679 19609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:30.537410 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.533199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33099 (local address 127.0.0.1:44154)
0812 03:52:30.533598 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.533609 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.533673 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.533951 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.533952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.533954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:30.534017 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.534019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.535026 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.535027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.536251 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.536254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.537080 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.537081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.537163 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.537177 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.537250 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.537310 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":86,"threads_started":1}
I20260812 03:52:30.537418 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.533308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44154 (local address 127.17.54.193:33099)
0812 03:52:30.533539 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.533544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.533753 (+ 209us) server_negotiation.cc:407] Connection header received
0812 03:52:30.533796 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.533798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.533815 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.533882 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:30.534106 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.534943 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.536343 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.536991 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.537080 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.537145 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.537278 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.537317 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":99,"threads_started":1}
I20260812 03:52:30.537720 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.533393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:38037 (local address 127.0.0.1:45156)
0812 03:52:30.533819 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.533827 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.533898 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.534259 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.534260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.534262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:30.534330 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.534332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.535554 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.535556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.536557 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.536561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.537486 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.537487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.537555 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.537560 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.537614 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.537653 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":72,"threads_started":1}
I20260812 03:52:30.538038 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.533564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.17.54.194:38037)
0812 03:52:30.533769 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.533774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.533995 (+ 221us) server_negotiation.cc:407] Connection header received
0812 03:52:30.534101 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.534103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.534120 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.534193 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:30.534413 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.535456 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.536653 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.537347 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.537407 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.537455 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.537873 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.537928 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
I20260812 03:52:30.538089 20166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.533197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:45881 (local address 127.0.0.1:60760)
0812 03:52:30.533531 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.533576 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.533662 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.533885 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.533888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.533891 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:30.533969 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.533972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.535376 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.535377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.536435 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.536437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.537455 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.537456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.537919 (+ 463us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.537923 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.537974 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.538017 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":49,"threads_started":1}
I20260812 03:52:30.538221 19762 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5fc083e11342969b167bfc72debf64 (DEFAULT_TABLE table=test-table [id=6ca04f0a864440d6965df6f3151cf12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:30.538409 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.533290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.17.54.195:45881)
0812 03:52:30.533586 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.533590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.533642 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:52:30.533757 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.533760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.533778 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.533866 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:30.534080 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.535277 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.536524 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.537271 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.537345 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.537415 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.538226 (+ 811us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.538274 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":100,"threads_started":1}
I20260812 03:52:30.538691 19762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5fc083e11342969b167bfc72debf64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:30.538712 19904 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5fc083e11342969b167bfc72debf64 (DEFAULT_TABLE table=test-table [id=6ca04f0a864440d6965df6f3151cf12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:30.539081 19904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5fc083e11342969b167bfc72debf64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:30.542145 20173 tablet_bootstrap.cc:492] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Bootstrap starting.
I20260812 03:52:30.542335 20174 tablet_bootstrap.cc:492] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Bootstrap starting.
I20260812 03:52:30.542881 20173 tablet_bootstrap.cc:654] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:30.542876 20064 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5fc083e11342969b167bfc72debf64 (DEFAULT_TABLE table=test-table [id=6ca04f0a864440d6965df6f3151cf12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:30.543038 20174 tablet_bootstrap.cc:654] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:30.543238 20173 log.cc:824] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:30.543262 20064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5fc083e11342969b167bfc72debf64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:30.543354 20174 log.cc:824] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:30.544135 20174 tablet_bootstrap.cc:492] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: No bootstrap required, opened a new log
I20260812 03:52:30.544143 20173 tablet_bootstrap.cc:492] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: No bootstrap required, opened a new log
I20260812 03:52:30.544219 20173 ts_tablet_manager.cc:1397] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:52:30.544222 20174 ts_tablet_manager.cc:1397] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:30.546154 20174 raft_consensus.cc:348] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.546347 20174 raft_consensus.cc:374] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:30.546401 20174 raft_consensus.cc:729] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5beed0f7b84d40da9270b2cea8842804, State: Initialized, Role: FOLLOWER
I20260812 03:52:30.546535 20174 consensus_queue.cc:260] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.546522 20173 raft_consensus.cc:348] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.546690 20173 raft_consensus.cc:374] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:30.546725 20173 raft_consensus.cc:729] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ad518a9ffb4452b0a9785b4e8ae419, State: Initialized, Role: FOLLOWER
I20260812 03:52:30.546866 20174 ts_tablet_manager.cc:1428] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 03:52:30.546859 20173 consensus_queue.cc:260] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.547005 19991 heartbeater.cc:503] Master 127.17.54.254:46381 was elected leader, sending a full tablet report...
I20260812 03:52:30.547029 20178 tablet_bootstrap.cc:492] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Bootstrap starting.
I20260812 03:52:30.547129 20173 ts_tablet_manager.cc:1428] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 03:52:30.547277 19837 heartbeater.cc:503] Master 127.17.54.254:46381 was elected leader, sending a full tablet report...
I20260812 03:52:30.548254 20178 tablet_bootstrap.cc:654] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:30.548619 20178 log.cc:824] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:30.549364 20178 tablet_bootstrap.cc:492] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: No bootstrap required, opened a new log
I20260812 03:52:30.549460 20178 ts_tablet_manager.cc:1397] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:52:30.550962 20178 raft_consensus.cc:348] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.551122 20178 raft_consensus.cc:374] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:30.551172 20178 raft_consensus.cc:729] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2bc4088aad94fa3b331075ab65e6b61, State: Initialized, Role: FOLLOWER
I20260812 03:52:30.551296 20178 consensus_queue.cc:260] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.551549 20178 ts_tablet_manager.cc:1428] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:52:30.551625 20146 heartbeater.cc:503] Master 127.17.54.254:46381 was elected leader, sending a full tablet report...
W20260812 03:52:30.561442 19992 tablet.cc:2369] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:52:30.636806 19838 tablet.cc:2369] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:30.704274 20181 raft_consensus.cc:482] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:30.704471 20181 raft_consensus.cc:504] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.704895 20181 leader_election.cc:288] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099), 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037)
I20260812 03:52:30.709337 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.705216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:44475 (local address 127.17.54.193:33099)
0812 03:52:30.705272 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.705275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.705587 (+ 312us) server_negotiation.cc:407] Connection header received
0812 03:52:30.705622 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.705623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.705630 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.705706 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:30.705856 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.706590 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.708195 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.708848 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.708946 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.708957 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.709199 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.709259 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:52:30.709337 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.705302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:37219 (local address 127.17.54.194:38037)
0812 03:52:30.705356 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.705359 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.705380 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:52:30.705424 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.705425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.705435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.705504 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:30.705698 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.706526 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.708191 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.708842 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.708947 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.708957 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.709199 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.709256 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:52:30.709327 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.705175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:38037 (local address 127.17.54.195:37219)
0812 03:52:30.705234 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.705248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.705322 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.705527 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.705528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.705530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:30.705618 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.705620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.706671 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.706672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.708064 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.708068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.708948 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.708949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.709059 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.709062 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.709137 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.709209 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":110}
I20260812 03:52:30.709334 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.705181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33099 (local address 127.17.54.195:44475)
0812 03:52:30.705508 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.705519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.705574 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.705713 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.705714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.705715 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:30.705771 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.705772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.706675 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.706676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.708064 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.708068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.708948 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.708949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.709052 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.709057 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.709135 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.709209 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":114,"thread_start_us":204,"threads_started":1}
I20260812 03:52:30.709883 19942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5fc083e11342969b167bfc72debf64" candidate_uuid: "f2bc4088aad94fa3b331075ab65e6b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5beed0f7b84d40da9270b2cea8842804" is_pre_election: true
I20260812 03:52:30.710106 19942 raft_consensus.cc:2454] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2bc4088aad94fa3b331075ab65e6b61 in term 0.
I20260812 03:52:30.710114 19784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5fc083e11342969b167bfc72debf64" candidate_uuid: "f2bc4088aad94fa3b331075ab65e6b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" is_pre_election: true
I20260812 03:52:30.710253 19784 raft_consensus.cc:2454] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2bc4088aad94fa3b331075ab65e6b61 in term 0.
I20260812 03:52:30.710440 20021 leader_election.cc:302] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5beed0f7b84d40da9270b2cea8842804, f2bc4088aad94fa3b331075ab65e6b61; no voters:
I20260812 03:52:30.710580 20181 raft_consensus.cc:2781] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:30.710638 20181 raft_consensus.cc:482] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:30.710664 20181 raft_consensus.cc:3037] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:30.711440 20181 raft_consensus.cc:504] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.711617 20181 leader_election.cc:288] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [CANDIDATE]: Term 1 election: Requested vote from peers 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099), 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037)
I20260812 03:52:30.711768 19784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5fc083e11342969b167bfc72debf64" candidate_uuid: "f2bc4088aad94fa3b331075ab65e6b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04ad518a9ffb4452b0a9785b4e8ae419"
I20260812 03:52:30.711798 19942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5fc083e11342969b167bfc72debf64" candidate_uuid: "f2bc4088aad94fa3b331075ab65e6b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5beed0f7b84d40da9270b2cea8842804"
I20260812 03:52:30.711872 19942 raft_consensus.cc:3037] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:30.711864 19784 raft_consensus.cc:3037] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:30.712836 19942 raft_consensus.cc:2454] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2bc4088aad94fa3b331075ab65e6b61 in term 1.
I20260812 03:52:30.712836 19784 raft_consensus.cc:2454] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2bc4088aad94fa3b331075ab65e6b61 in term 1.
I20260812 03:52:30.713023 20021 leader_election.cc:302] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5beed0f7b84d40da9270b2cea8842804, f2bc4088aad94fa3b331075ab65e6b61; no voters:
I20260812 03:52:30.713126 20181 raft_consensus.cc:2781] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:30.713294 20181 raft_consensus.cc:686] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 1 LEADER]: Becoming Leader. State: Replica: f2bc4088aad94fa3b331075ab65e6b61, State: Running, Role: LEADER
I20260812 03:52:30.713404 20181 consensus_queue.cc:237] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } }
I20260812 03:52:30.714421 19607 catalog_manager.cc:5495] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "f2bc4088aad94fa3b331075ab65e6b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } health_report { overall_health: UNKNOWN } } }
W20260812 03:52:30.739850 20147 tablet.cc:2369] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:52:30 dist-test-slave-tm1g krb5kdc[19559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506749, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:30.782080 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.775086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:46381 (local address 127.0.0.1:46236)
0812 03:52:30.775236 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.775248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.775308 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.775468 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.775468 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.775782 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:30.775836 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.775839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.776515 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.776516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.777041 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.777042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.777379 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.777380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.777430 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.777992 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:30.777997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:30.779461 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:30.781111 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.781119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.781121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.781314 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.781521 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:30.781522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:30.781522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:30.781581 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:30.781789 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:30.781795 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:30.781836 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.781887 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.781986 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260812 03:52:30.782131 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.775179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46236 (local address 127.17.54.254:46381)
0812 03:52:30.775281 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.775283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.775310 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:52:30.775372 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.775372 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.775382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.775427 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:30.775922 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.776439 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.777124 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.777308 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.777365 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.779571 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:30.779572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:30.779573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:30.779579 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:30.781023 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.781401 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.781401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.781402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.781458 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:30.781633 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:30.781633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:30.781633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:30.781735 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:30.781790 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.781917 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.782034 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":72}
I20260812 03:52:30.784332 19607 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:46236:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:52:30.784534 19607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:30.786129 19762 tablet_service.cc:1467] Processing CreateTablet for tablet 56b3180894d04cf78dd2418b850513d5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb4132ff32ef411da40e9abf9e5f2ae2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:30.786129 19904 tablet_service.cc:1467] Processing CreateTablet for tablet 56b3180894d04cf78dd2418b850513d5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb4132ff32ef411da40e9abf9e5f2ae2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:30.786180 20064 tablet_service.cc:1467] Processing CreateTablet for tablet 56b3180894d04cf78dd2418b850513d5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb4132ff32ef411da40e9abf9e5f2ae2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:30.786355 19904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b3180894d04cf78dd2418b850513d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:30.786356 19762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b3180894d04cf78dd2418b850513d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:30.786389 20064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b3180894d04cf78dd2418b850513d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:30.787918 20178 tablet_bootstrap.cc:492] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: Bootstrap starting.
I20260812 03:52:30.787925 20174 tablet_bootstrap.cc:492] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: Bootstrap starting.
I20260812 03:52:30.787953 20173 tablet_bootstrap.cc:492] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: Bootstrap starting.
I20260812 03:52:30.788281 20173 tablet_bootstrap.cc:654] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:30.788436 20178 tablet_bootstrap.cc:654] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:30.788677 20174 tablet_bootstrap.cc:654] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:30.788899 20173 tablet_bootstrap.cc:492] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: No bootstrap required, opened a new log
I20260812 03:52:30.788949 20173 ts_tablet_manager.cc:1397] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:30.789152 20178 tablet_bootstrap.cc:492] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: No bootstrap required, opened a new log
I20260812 03:52:30.789111 20173 raft_consensus.cc:348] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.789194 20173 raft_consensus.cc:374] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:30.789201 20178 ts_tablet_manager.cc:1397] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:30.789220 20173 raft_consensus.cc:729] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ad518a9ffb4452b0a9785b4e8ae419, State: Initialized, Role: FOLLOWER
I20260812 03:52:30.789281 20173 consensus_queue.cc:260] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.789395 20173 ts_tablet_manager.cc:1428] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.789356 20178 raft_consensus.cc:348] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.789430 20178 raft_consensus.cc:374] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:30.789414 20174 tablet_bootstrap.cc:492] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: No bootstrap required, opened a new log
I20260812 03:52:30.789461 20178 raft_consensus.cc:729] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2bc4088aad94fa3b331075ab65e6b61, State: Initialized, Role: FOLLOWER
I20260812 03:52:30.789476 20174 ts_tablet_manager.cc:1397] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 03:52:30.789501 20178 consensus_queue.cc:260] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.789588 20178 ts_tablet_manager.cc:1428] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.789553 20179 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:30.789631 20179 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:30.789644 20174 raft_consensus.cc:348] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.789723 20174 raft_consensus.cc:374] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:30.789757 20174 raft_consensus.cc:729] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5beed0f7b84d40da9270b2cea8842804, State: Initialized, Role: FOLLOWER
I20260812 03:52:30.789799 20174 consensus_queue.cc:260] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.789893 20174 ts_tablet_manager.cc:1428] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:30.789868 20181 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:30.789942 20181 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:30.789974 20177 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:30.790071 20177 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:30.968638 20177 raft_consensus.cc:482] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:30.968747 20177 raft_consensus.cc:504] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.969168 20177 leader_election.cc:288] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099), f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:30.972261 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.969665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:49479 (local address 127.17.54.195:45881)
0812 03:52:30.969710 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.969713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.969722 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:30.969781 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.969782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.969792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.969873 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:30.970016 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.970529 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.971529 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.971888 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.971983 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.971993 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.972147 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.972187 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:30.972270 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.969445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:56625 (local address 127.17.54.193:33099)
0812 03:52:30.969497 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:30.969499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:30.969605 (+ 106us) server_negotiation.cc:407] Connection header received
0812 03:52:30.969721 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:30.969722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:30.969732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:30.969764 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:30.970041 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.970582 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.971551 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.971905 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.971998 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.972006 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:30.972162 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 03:52:30.972202 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:30.972295 20196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.969517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:45881 (local address 127.17.54.194:49479)
0812 03:52:30.969654 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.969664 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.969720 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.969879 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.969880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.969882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:30.969955 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.969957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.970621 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.970622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.971430 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.971432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.971983 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.971984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.972057 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.972060 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.972138 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.972184 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":27,"thread_start_us":54,"threads_started":1}
I20260812 03:52:30.972299 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:30.969374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33099 (local address 127.17.54.194:56625)
0812 03:52:30.969534 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:30.969546 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:30.969639 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:30.969840 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:30.969840 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:30.969843 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:30.969933 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.969935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.970666 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.970667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.971417 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:30.971448 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:30.971998 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:30.971998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:30.972080 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:30.972083 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:30.972152 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:52:30.972200 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":7,"thread_start_us":79,"threads_started":1}
I20260812 03:52:30.972620 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b3180894d04cf78dd2418b850513d5" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2bc4088aad94fa3b331075ab65e6b61" is_pre_election: true
I20260812 03:52:30.972667 19784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b3180894d04cf78dd2418b850513d5" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" is_pre_election: true
I20260812 03:52:30.972743 20099 raft_consensus.cc:2454] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 0.
I20260812 03:52:30.972790 19784 raft_consensus.cc:2454] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 0.
I20260812 03:52:30.972950 19866 leader_election.cc:302] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5beed0f7b84d40da9270b2cea8842804, f2bc4088aad94fa3b331075ab65e6b61; no voters:
I20260812 03:52:30.973106 20177 raft_consensus.cc:2781] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:30.973171 20177 raft_consensus.cc:482] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:30.973202 20177 raft_consensus.cc:3037] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:30.973953 20177 raft_consensus.cc:504] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.974103 20177 leader_election.cc:288] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 election: Requested vote from peers 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099), f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:30.974260 19784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b3180894d04cf78dd2418b850513d5" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04ad518a9ffb4452b0a9785b4e8ae419"
I20260812 03:52:30.974249 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b3180894d04cf78dd2418b850513d5" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2bc4088aad94fa3b331075ab65e6b61"
I20260812 03:52:30.974375 19784 raft_consensus.cc:3037] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:30.974375 20099 raft_consensus.cc:3037] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:30.974982 19784 raft_consensus.cc:2454] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 1.
I20260812 03:52:30.975160 20099 raft_consensus.cc:2454] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 1.
I20260812 03:52:30.975234 19869 leader_election.cc:302] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ad518a9ffb4452b0a9785b4e8ae419, 5beed0f7b84d40da9270b2cea8842804; no voters:
I20260812 03:52:30.975322 20177 raft_consensus.cc:2781] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:30.975447 20177 raft_consensus.cc:686] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [term 1 LEADER]: Becoming Leader. State: Replica: 5beed0f7b84d40da9270b2cea8842804, State: Running, Role: LEADER
I20260812 03:52:30.975554 20177 consensus_queue.cc:237] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:30.976114 20197 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: TxnStatusTablet state changed. Reason: New leader 5beed0f7b84d40da9270b2cea8842804. Latest consensus state: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:30.976209 20197 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:30.976397 20200 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:52:30.976533 20200 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:52:30.976508 19607 catalog_manager.cc:5495] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194). New cstate: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:31.020438 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.017354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45168 (local address 127.17.54.194:38037)
0812 03:52:31.017422 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.017424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.017521 (+ 97us) server_negotiation.cc:407] Connection header received
0812 03:52:31.017610 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.017611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.017629 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.017686 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:31.017819 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.018385 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.019577 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.019794 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.019857 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.020065 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.020175 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.020237 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.020318 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.020338 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:52:31.020468 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.017274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:38037 (local address 127.0.0.1:45168)
0812 03:52:31.017437 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:31.017450 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:31.017521 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:31.017693 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:31.017694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:31.017695 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:31.017753 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.017756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.018461 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.018462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.019475 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.019480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.019862 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.019863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.019941 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.019964 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.020246 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.020255 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:52:31.020316 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:52:31.020379 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 03:52:31.022475 20099 raft_consensus.cc:1261] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Refusing update from remote peer 5beed0f7b84d40da9270b2cea8842804: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:31.022487 19784 raft_consensus.cc:1261] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Refusing update from remote peer 5beed0f7b84d40da9270b2cea8842804: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:31.022836 20177 consensus_queue.cc:1035] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:31.023051 20197 consensus_queue.cc:1035] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:31.024062 20181 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: TxnStatusTablet state changed. Reason: New leader 5beed0f7b84d40da9270b2cea8842804. Latest consensus state: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:31.024161 20181 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:31.024991 20177 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:31.025087 20177 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:31.025192 20203 mvcc.cc:205] Tried to move back new op lower bound from 7317531652185038848 to 7317531651996524544. Current Snapshot: MvccSnapshot[applied={T|T < 7317531652185038848}]
I20260812 03:52:31.025656 20179 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: TxnStatusTablet state changed. Reason: New leader 5beed0f7b84d40da9270b2cea8842804. Latest consensus state: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:31.025756 20179 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:31.026778 20181 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:31.026883 20181 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P f2bc4088aad94fa3b331075ab65e6b61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:31.027259 20204 mvcc.cc:205] Tried to move back new op lower bound from 7317531652185038848 to 7317531651996524544. Current Snapshot: MvccSnapshot[applied={T|T < 7317531652185038848 or (T in {7317531652185038848})}]
I20260812 03:52:31.028316 20197 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:31.028406 20197 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P 5beed0f7b84d40da9270b2cea8842804: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:31.028941 20179 tablet_replica.cc:439] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } } }
I20260812 03:52:31.029057 20179 tablet_replica.cc:442] T 56b3180894d04cf78dd2418b850513d5 P 04ad518a9ffb4452b0a9785b4e8ae419: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:31.028990 20208 mvcc.cc:205] Tried to move back new op lower bound from 7317531652185038848 to 7317531651996524544. Current Snapshot: MvccSnapshot[applied={T|T < 7317531652185038848}]
I20260812 03:52:31.031170 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.027510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60770 (local address 127.17.54.195:45881)
0812 03:52:31.027592 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.027594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.027749 (+ 155us) server_negotiation.cc:407] Connection header received
0812 03:52:31.027826 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.027827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.027844 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.027912 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:31.028066 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.028776 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.030162 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.030483 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.030549 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.030759 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.030898 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.030989 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.031071 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.031099 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:52:31.037074 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.033960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:54023 (local address 127.17.54.194:38037)
0812 03:52:31.034013 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.034015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.034287 (+ 272us) server_negotiation.cc:407] Connection header received
0812 03:52:31.034387 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.034388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.034398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.034452 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:31.034654 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.035180 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.036215 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.036431 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.036495 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.036812 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.036847 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.036905 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.036997 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.037016 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:52:31.037118 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.033855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:38037 (local address 127.17.54.195:54023)
0812 03:52:31.034165 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:31.034178 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:31.034252 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:31.034473 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:31.034474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:31.034476 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:31.034545 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.034546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.035254 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.035255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.036117 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.036120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.036516 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.036517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.036586 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.036603 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.036924 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.036926 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:52:31.036974 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 03:52:31.037024 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":124,"threads_started":1}
I20260812 03:52:31.040038 19784 raft_consensus.cc:1261] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Refusing update from remote peer f2bc4088aad94fa3b331075ab65e6b61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:31.040153 19942 raft_consensus.cc:1261] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Refusing update from remote peer f2bc4088aad94fa3b331075ab65e6b61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:31.040349 20183 consensus_queue.cc:1035] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:31.040503 20181 consensus_queue.cc:1035] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:31.050266 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.047290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:34627 (local address 127.17.54.195:45881)
0812 03:52:31.047355 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.047357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.047367 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:31.047391 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.047392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.047403 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.047467 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:31.047604 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.048335 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.049507 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.049806 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.049890 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.050029 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.050064 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.050120 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.050165 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.050189 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:52:31.050262 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.047062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:45881 (local address 127.17.54.194:34627)
0812 03:52:31.047250 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:31.047262 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:31.047319 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:31.047479 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:31.047480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:31.047484 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:31.047533 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.047534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.048411 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.048412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.049428 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.049431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.049892 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.049893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.049940 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.049954 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.050126 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.050127 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:52:31.050161 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 03:52:31.050200 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260812 03:52:31.059319 19607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:31.059463 19607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:31.060228 19607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: Sending DeleteTablet for 3 replicas of tablet 6e5fc083e11342969b167bfc72debf64
I20260812 03:52:31.060591 19762 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5fc083e11342969b167bfc72debf64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:31 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:44154
I20260812 03:52:31.060668 20064 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5fc083e11342969b167bfc72debf64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:31 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:60760
I20260812 03:52:31.060945 19904 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5fc083e11342969b167bfc72debf64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:31 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:45156
I20260812 03:52:31.061307 19607 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:31.061483 19607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:31.061738 20227 tablet_replica.cc:330] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: stopping tablet replica
I20260812 03:52:31.062752 19762 tablet_service.cc:1467] Processing CreateTablet for tablet 0177391a40e74f3abfd26823160bccfc (DEFAULT_TABLE table=test-table [id=b4227e648b0c40cb9c4f9fa4aa994f51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:31.062762 19904 tablet_service.cc:1467] Processing CreateTablet for tablet 0177391a40e74f3abfd26823160bccfc (DEFAULT_TABLE table=test-table [id=b4227e648b0c40cb9c4f9fa4aa994f51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:31.062830 20064 tablet_service.cc:1467] Processing CreateTablet for tablet 0177391a40e74f3abfd26823160bccfc (DEFAULT_TABLE table=test-table [id=b4227e648b0c40cb9c4f9fa4aa994f51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:31.062903 19904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0177391a40e74f3abfd26823160bccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:31.062918 19762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0177391a40e74f3abfd26823160bccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:31.062932 20064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0177391a40e74f3abfd26823160bccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:31.064282 20178 tablet_bootstrap.cc:492] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: Bootstrap starting.
I20260812 03:52:31.064309 20174 tablet_bootstrap.cc:492] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: Bootstrap starting.
I20260812 03:52:31.064412 20173 tablet_bootstrap.cc:492] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: Bootstrap starting.
I20260812 03:52:31.064738 20178 tablet_bootstrap.cc:654] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:31.064806 20174 tablet_bootstrap.cc:654] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:31.064816 20173 tablet_bootstrap.cc:654] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:31.065694 20178 tablet_bootstrap.cc:492] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: No bootstrap required, opened a new log
I20260812 03:52:31.065773 20174 tablet_bootstrap.cc:492] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: No bootstrap required, opened a new log
I20260812 03:52:31.065778 20178 ts_tablet_manager.cc:1397] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:31.065829 20174 ts_tablet_manager.cc:1397] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 03:52:31.065954 20178 raft_consensus.cc:348] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.065985 20174 raft_consensus.cc:348] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.066051 20174 raft_consensus.cc:374] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:31.066053 20178 raft_consensus.cc:374] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:31.066072 20174 raft_consensus.cc:729] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5beed0f7b84d40da9270b2cea8842804, State: Initialized, Role: FOLLOWER
I20260812 03:52:31.066085 20178 raft_consensus.cc:729] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2bc4088aad94fa3b331075ab65e6b61, State: Initialized, Role: FOLLOWER
I20260812 03:52:31.066118 20174 consensus_queue.cc:260] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.066123 20178 consensus_queue.cc:260] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.066210 20174 ts_tablet_manager.cc:1428] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:31.066223 20178 ts_tablet_manager.cc:1428] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:31.066435 20227 raft_consensus.cc:2229] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:31.066593 20227 raft_consensus.cc:2258] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:31.070649 20225 tablet_replica.cc:330] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: stopping tablet replica
I20260812 03:52:31.070734 20173 tablet_bootstrap.cc:492] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: No bootstrap required, opened a new log
I20260812 03:52:31.070784 20173 ts_tablet_manager.cc:1397] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20260812 03:52:31.070922 20225 raft_consensus.cc:2229] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:31.070940 20173 raft_consensus.cc:348] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.071053 20225 raft_consensus.cc:2258] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:31.071061 20173 raft_consensus.cc:374] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:31.071171 20173 raft_consensus.cc:729] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ad518a9ffb4452b0a9785b4e8ae419, State: Initialized, Role: FOLLOWER
I20260812 03:52:31.071228 20173 consensus_queue.cc:260] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.071328 20173 ts_tablet_manager.cc:1428] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:31.072000 20225 ts_tablet_manager.cc:1905] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:31.073951 20225 ts_tablet_manager.cc:1918] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:31.074064 20225 log.cc:1196] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/6e5fc083e11342969b167bfc72debf64
I20260812 03:52:31.074440 20226 tablet_replica.cc:330] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: stopping tablet replica
I20260812 03:52:31.074460 20225 ts_tablet_manager.cc:1939] T 6e5fc083e11342969b167bfc72debf64 P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting consensus metadata
I20260812 03:52:31.074595 20227 ts_tablet_manager.cc:1905] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:31.074652 20226 raft_consensus.cc:2229] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:31.074903 20226 raft_consensus.cc:2258] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:31.074932 19592 catalog_manager.cc:4841] TS 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099): tablet 6e5fc083e11342969b167bfc72debf64 (table test-table [id=6ca04f0a864440d6965df6f3151cf12f]) successfully deleted
I20260812 03:52:31.075361 20226 ts_tablet_manager.cc:1905] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:31.076462 20227 ts_tablet_manager.cc:1918] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:31.076606 20227 log.cc:1196] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/6e5fc083e11342969b167bfc72debf64
I20260812 03:52:31.076992 20227 ts_tablet_manager.cc:1939] T 6e5fc083e11342969b167bfc72debf64 P 5beed0f7b84d40da9270b2cea8842804: Deleting consensus metadata
I20260812 03:52:31.077174 20226 ts_tablet_manager.cc:1918] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:31.077253 20226 log.cc:1196] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/6e5fc083e11342969b167bfc72debf64
I20260812 03:52:31.077528 20226 ts_tablet_manager.cc:1939] T 6e5fc083e11342969b167bfc72debf64 P f2bc4088aad94fa3b331075ab65e6b61: Deleting consensus metadata
I20260812 03:52:31.077608 19590 catalog_manager.cc:4841] TS 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037): tablet 6e5fc083e11342969b167bfc72debf64 (table test-table [id=6ca04f0a864440d6965df6f3151cf12f]) successfully deleted
I20260812 03:52:31.077984 19590 catalog_manager.cc:4841] TS f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881): tablet 6e5fc083e11342969b167bfc72debf64 (table test-table [id=6ca04f0a864440d6965df6f3151cf12f]) successfully deleted
I20260812 03:52:31.257036 20179 raft_consensus.cc:482] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:31.257162 20179 raft_consensus.cc:504] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.257545 20179 leader_election.cc:288] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037), f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:31.261826 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.257746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:38037 (local address 127.17.54.193:44305)
0812 03:52:31.257882 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:31.257894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:31.257972 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:31.258369 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:31.258371 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:31.258374 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:31.258463 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.258466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.259363 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.259364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.260721 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.260725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.261547 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.261549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.261618 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.261624 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:52:31.261674 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:52:31.261717 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260812 03:52:31.261837 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.257933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:44305 (local address 127.17.54.194:38037)
0812 03:52:31.258010 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.258013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.258022 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:31.258052 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.258053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.258145 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.258259 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:31.258550 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.259271 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.260828 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.261463 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.261539 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.261548 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.261687 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.261743 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:52:31.262077 19942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0177391a40e74f3abfd26823160bccfc" candidate_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5beed0f7b84d40da9270b2cea8842804" is_pre_election: true
I20260812 03:52:31.262162 19942 raft_consensus.cc:2454] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04ad518a9ffb4452b0a9785b4e8ae419 in term 0.
I20260812 03:52:31.262230 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.258290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:40565 (local address 127.17.54.195:45881)
0812 03:52:31.258406 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.258408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.258417 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:31.258444 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.258445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.258458 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.258521 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:31.259123 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.259832 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.261179 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.261758 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.261831 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.261839 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.262118 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.262168 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:52:31.262245 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.257946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:45881 (local address 127.17.54.193:40565)
0812 03:52:31.258116 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:31.258126 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:31.258168 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:31.258957 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:31.258959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:31.258962 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:31.259022 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.259024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.259978 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.259979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.261100 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.261103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.261847 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.261848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.261993 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.261997 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:31.262129 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 03:52:31.262166 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260812 03:52:31.262369 19707 leader_election.cc:302] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ad518a9ffb4452b0a9785b4e8ae419, 5beed0f7b84d40da9270b2cea8842804; no voters:
I20260812 03:52:31.262544 20179 raft_consensus.cc:2781] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:31.262599 20179 raft_consensus.cc:482] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:31.262629 20179 raft_consensus.cc:3037] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:31.262676 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0177391a40e74f3abfd26823160bccfc" candidate_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2bc4088aad94fa3b331075ab65e6b61" is_pre_election: true
I20260812 03:52:31.262777 20099 raft_consensus.cc:2454] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04ad518a9ffb4452b0a9785b4e8ae419 in term 0.
I20260812 03:52:31.263448 20179 raft_consensus.cc:504] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.263558 20179 leader_election.cc:288] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [CANDIDATE]: Term 1 election: Requested vote from peers 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037), f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:31.263846 19942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0177391a40e74f3abfd26823160bccfc" candidate_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5beed0f7b84d40da9270b2cea8842804"
I20260812 03:52:31.263866 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0177391a40e74f3abfd26823160bccfc" candidate_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2bc4088aad94fa3b331075ab65e6b61"
I20260812 03:52:31.263939 19942 raft_consensus.cc:3037] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:31.263960 20099 raft_consensus.cc:3037] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:31.264605 19942 raft_consensus.cc:2454] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04ad518a9ffb4452b0a9785b4e8ae419 in term 1.
I20260812 03:52:31.264602 20099 raft_consensus.cc:2454] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04ad518a9ffb4452b0a9785b4e8ae419 in term 1.
I20260812 03:52:31.264777 19707 leader_election.cc:302] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ad518a9ffb4452b0a9785b4e8ae419, f2bc4088aad94fa3b331075ab65e6b61; no voters:
I20260812 03:52:31.264885 20179 raft_consensus.cc:2781] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:31.265065 20179 raft_consensus.cc:686] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 LEADER]: Becoming Leader. State: Replica: 04ad518a9ffb4452b0a9785b4e8ae419, State: Running, Role: LEADER
I20260812 03:52:31.265125 20179 consensus_queue.cc:237] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:31.265974 19609 catalog_manager.cc:5495] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193). New cstate: current_term: 1 leader_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:31.306599 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.304068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.17.54.193:33099)
0812 03:52:31.304125 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.304127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.304160 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:52:31.304260 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.304261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.304286 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.304380 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:31.304500 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.304995 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.305750 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.305966 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.306042 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.306225 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.306361 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.306456 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.306480 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.306504 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:52:31.312315 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.309681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:54023 (local address 127.17.54.194:38037)
0812 03:52:31.309721 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.309723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.309838 (+ 115us) server_negotiation.cc:407] Connection header received
0812 03:52:31.309876 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.309877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.309885 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.309931 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:31.310028 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.310676 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.311420 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.311723 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.311797 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.312011 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.312046 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.312098 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.312206 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.312233 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:52:31.312357 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.309600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:38037 (local address 127.17.54.193:54023)
0812 03:52:31.309760 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:31.309771 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:31.309834 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:31.309932 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:31.309932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:31.309933 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:31.309992 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.309993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.310768 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.310768 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.311336 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.311338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.311844 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.311845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.311903 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.311924 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.312111 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.312113 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:52:31.312185 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:52:31.312235 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260812 03:52:31.314391 20099 raft_consensus.cc:1261] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Refusing update from remote peer 04ad518a9ffb4452b0a9785b4e8ae419: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:31.314429 19942 raft_consensus.cc:1261] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Refusing update from remote peer 04ad518a9ffb4452b0a9785b4e8ae419: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:31.314647 20179 consensus_queue.cc:1035] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:31.314778 20235 consensus_queue.cc:1035] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:31.322248 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.319853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:60725 (local address 127.17.54.193:33099)
0812 03:52:31.319934 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:31.319936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:31.319944 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:31.319967 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:31.319968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:31.319978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:31.320022 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:31.320130 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.320606 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.321413 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.321736 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.321802 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.321994 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.322028 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.322071 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:31.322153 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:52:31.322175 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:52:31.322602 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:31.319673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33099 (local address 127.17.54.194:60725)
0812 03:52:31.319733 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:31.319745 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:31.319791 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:31.320039 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:31.320039 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:31.320041 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:31.320083 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.320084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.320693 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.320694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.321270 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:31.321272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:31.321840 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:31.321841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:31.321916 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:31.321924 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:31.322075 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:31.322077 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:52:31.322157 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:52:31.322210 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 03:52:31.322934 20208 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:52:32.844244 19609 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:32.844358 19609 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:32.845533 19609 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: Sending DeleteTablet for 3 replicas of tablet 0177391a40e74f3abfd26823160bccfc
I20260812 03:52:32.845825 19904 tablet_service.cc:1514] Processing DeleteTablet for tablet 0177391a40e74f3abfd26823160bccfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:32 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:45156
I20260812 03:52:32.845832 19762 tablet_service.cc:1514] Processing DeleteTablet for tablet 0177391a40e74f3abfd26823160bccfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:32 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:44154
I20260812 03:52:32.845857 20064 tablet_service.cc:1514] Processing DeleteTablet for tablet 0177391a40e74f3abfd26823160bccfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:32 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:60760
I20260812 03:52:32.846074 20268 tablet_replica.cc:330] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: stopping tablet replica
I20260812 03:52:32.846139 20268 raft_consensus.cc:2229] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:32.846236 20268 raft_consensus.cc:2258] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:32.846474 20270 tablet_replica.cc:330] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: stopping tablet replica
I20260812 03:52:32.846554 20270 raft_consensus.cc:2229] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:32.846553 20268 ts_tablet_manager.cc:1905] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:32.846603 20270 raft_consensus.cc:2258] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:32.846728 20267 tablet_replica.cc:330] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: stopping tablet replica
I20260812 03:52:32.846805 20267 raft_consensus.cc:2229] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:32.846850 20267 raft_consensus.cc:2258] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:32.846894 20270 ts_tablet_manager.cc:1905] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:32.848932 20267 ts_tablet_manager.cc:1905] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:32.848986 20270 ts_tablet_manager.cc:1918] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:52:32.849061 20270 log.cc:1196] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/0177391a40e74f3abfd26823160bccfc
I20260812 03:52:32.849318 20270 ts_tablet_manager.cc:1939] T 0177391a40e74f3abfd26823160bccfc P f2bc4088aad94fa3b331075ab65e6b61: Deleting consensus metadata
I20260812 03:52:32.850443 20267 ts_tablet_manager.cc:1918] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:52:32.850513 20267 log.cc:1196] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/0177391a40e74f3abfd26823160bccfc
I20260812 03:52:32.850752 20267 ts_tablet_manager.cc:1939] T 0177391a40e74f3abfd26823160bccfc P 5beed0f7b84d40da9270b2cea8842804: Deleting consensus metadata
I20260812 03:52:32.850756 19590 catalog_manager.cc:4841] TS f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881): tablet 0177391a40e74f3abfd26823160bccfc (table test-table [id=b4227e648b0c40cb9c4f9fa4aa994f51]) successfully deleted
I20260812 03:52:32.851289 19590 catalog_manager.cc:4841] TS 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037): tablet 0177391a40e74f3abfd26823160bccfc (table test-table [id=b4227e648b0c40cb9c4f9fa4aa994f51]) successfully deleted
I20260812 03:52:32.851636 20268 ts_tablet_manager.cc:1918] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:52:32.851704 20268 log.cc:1196] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/0177391a40e74f3abfd26823160bccfc
I20260812 03:52:32.851974 20268 ts_tablet_manager.cc:1939] T 0177391a40e74f3abfd26823160bccfc P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting consensus metadata
I20260812 03:52:32.852341 19592 catalog_manager.cc:4841] TS 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099): tablet 0177391a40e74f3abfd26823160bccfc (table test-table [id=b4227e648b0c40cb9c4f9fa4aa994f51]) successfully deleted
Aug 12 03:52:32 dist-test-slave-tm1g krb5kdc[19559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:52:32 dist-test-slave-tm1g krb5kdc[19559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:32.872788 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:32.864629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.17.54.254:46381)
0812 03:52:32.864801 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:32.864803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:32.864814 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:32.864854 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:32.864855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:32.864866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:32.864918 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:32.865558 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:32.866204 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:32.867038 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:32.867238 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:32.867295 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:32.870178 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:32.870179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:32.870179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:32.870186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:32.871754 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:32.872114 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:32.872115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:32.872115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:32.872171 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:32.872340 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:32.872340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:32.872341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:32.872447 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:32.872528 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:32.872643 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 03:52:32.872713 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
I20260812 03:52:32.874441 19609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:32.874604 19609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:32.876130 19762 tablet_service.cc:1467] Processing CreateTablet for tablet befa592cc457488184429de5d3f444ef (DEFAULT_TABLE table=test-table [id=919eac0c7a184b12ad530c43ab503769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:32.876144 19904 tablet_service.cc:1467] Processing CreateTablet for tablet befa592cc457488184429de5d3f444ef (DEFAULT_TABLE table=test-table [id=919eac0c7a184b12ad530c43ab503769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:32.876170 20064 tablet_service.cc:1467] Processing CreateTablet for tablet befa592cc457488184429de5d3f444ef (DEFAULT_TABLE table=test-table [id=919eac0c7a184b12ad530c43ab503769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:32.876288 20064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befa592cc457488184429de5d3f444ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:32.876288 19762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befa592cc457488184429de5d3f444ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:32.876303 19904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befa592cc457488184429de5d3f444ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:32.877430 20279 tablet_bootstrap.cc:492] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: Bootstrap starting.
I20260812 03:52:32.877703 20280 tablet_bootstrap.cc:492] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: Bootstrap starting.
I20260812 03:52:32.877775 20281 tablet_bootstrap.cc:492] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: Bootstrap starting.
I20260812 03:52:32.878010 20279 tablet_bootstrap.cc:654] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:32.878057 20280 tablet_bootstrap.cc:654] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:32.878203 20281 tablet_bootstrap.cc:654] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:32.878670 20279 tablet_bootstrap.cc:492] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: No bootstrap required, opened a new log
I20260812 03:52:32.878670 20280 tablet_bootstrap.cc:492] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: No bootstrap required, opened a new log
I20260812 03:52:32.878727 20279 ts_tablet_manager.cc:1397] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:32.878727 20280 ts_tablet_manager.cc:1397] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:32.878849 20281 tablet_bootstrap.cc:492] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: No bootstrap required, opened a new log
I20260812 03:52:32.878862 20280 raft_consensus.cc:348] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.878876 20279 raft_consensus.cc:348] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.878932 20280 raft_consensus.cc:374] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:32.878940 20279 raft_consensus.cc:374] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:32.878937 20281 ts_tablet_manager.cc:1397] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:32.878966 20280 raft_consensus.cc:729] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ad518a9ffb4452b0a9785b4e8ae419, State: Initialized, Role: FOLLOWER
I20260812 03:52:32.878969 20279 raft_consensus.cc:729] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5beed0f7b84d40da9270b2cea8842804, State: Initialized, Role: FOLLOWER
I20260812 03:52:32.879015 20280 consensus_queue.cc:260] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.879015 20279 consensus_queue.cc:260] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.879125 20279 ts_tablet_manager.cc:1428] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:32.879140 20280 ts_tablet_manager.cc:1428] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:52:32.879071 20281 raft_consensus.cc:348] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.879307 20281 raft_consensus.cc:374] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:32.879338 20281 raft_consensus.cc:729] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2bc4088aad94fa3b331075ab65e6b61, State: Initialized, Role: FOLLOWER
I20260812 03:52:32.879405 20281 consensus_queue.cc:260] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.879580 20281 ts_tablet_manager.cc:1428] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:32.916945 20247 raft_consensus.cc:482] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:32.917058 20247 raft_consensus.cc:504] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.917232 20247 leader_election.cc:288] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099), f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:32.917457 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befa592cc457488184429de5d3f444ef" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2bc4088aad94fa3b331075ab65e6b61" is_pre_election: true
I20260812 03:52:32.917475 19784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befa592cc457488184429de5d3f444ef" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" is_pre_election: true
I20260812 03:52:32.917580 20099 raft_consensus.cc:2454] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 0.
I20260812 03:52:32.917577 19784 raft_consensus.cc:2454] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 0.
I20260812 03:52:32.917773 19866 leader_election.cc:302] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5beed0f7b84d40da9270b2cea8842804, f2bc4088aad94fa3b331075ab65e6b61; no voters:
I20260812 03:52:32.917910 20247 raft_consensus.cc:2781] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:32.917968 20247 raft_consensus.cc:482] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:32.917984 20247 raft_consensus.cc:3037] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:32.918910 20247 raft_consensus.cc:504] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.919075 20247 leader_election.cc:288] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 election: Requested vote from peers 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099), f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:32.919224 19784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befa592cc457488184429de5d3f444ef" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04ad518a9ffb4452b0a9785b4e8ae419"
I20260812 03:52:32.919312 19784 raft_consensus.cc:3037] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:32.919243 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befa592cc457488184429de5d3f444ef" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2bc4088aad94fa3b331075ab65e6b61"
I20260812 03:52:32.919348 20099 raft_consensus.cc:3037] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:32.920006 20099 raft_consensus.cc:2454] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 1.
I20260812 03:52:32.920054 19784 raft_consensus.cc:2454] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 1.
I20260812 03:52:32.920197 19866 leader_election.cc:302] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5beed0f7b84d40da9270b2cea8842804, f2bc4088aad94fa3b331075ab65e6b61; no voters:
I20260812 03:52:32.920306 20247 raft_consensus.cc:2781] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:32.920377 20247 raft_consensus.cc:686] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 1 LEADER]: Becoming Leader. State: Replica: 5beed0f7b84d40da9270b2cea8842804, State: Running, Role: LEADER
I20260812 03:52:32.920433 20247 consensus_queue.cc:237] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.921116 19607 catalog_manager.cc:5495] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194). New cstate: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:32.929983 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:32.927514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45178 (local address 127.17.54.194:38037)
0812 03:52:32.927660 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:32.927661 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:32.927667 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:52:32.927687 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:32.927687 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:32.927696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:32.927776 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:32.927952 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:32.928469 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:32.929239 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:32.929464 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:32.929542 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:32.929705 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:32.929747 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:32.929808 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:32.929867 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 03:52:32.929891 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260812 03:52:32.934489 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:32.932314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:43981 (local address 127.17.54.194:38037)
0812 03:52:32.932350 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:32.932352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:32.932360 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:32.932387 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:32.932387 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:32.932396 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:32.932441 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:32.932542 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:32.932971 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:32.933699 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:32.933887 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:32.933949 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:32.934169 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:32.934195 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:32.934238 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:32.934372 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 03:52:32.934396 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:52:32.934571 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:32.932130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:38037 (local address 127.17.54.194:43981)
0812 03:52:32.932275 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:32.932286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:32.932336 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:32.932456 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:32.932457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:32.932458 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:32.932498 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:32.932499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:32.933030 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:32.933030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:32.933609 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:32.933611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:32.933987 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:32.933988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:32.934074 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:32.934080 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:32.934265 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:32.934268 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:32.934370 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:52:32.934427 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260812 03:52:32.936548 19784 raft_consensus.cc:1261] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Refusing update from remote peer 5beed0f7b84d40da9270b2cea8842804: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:32.936594 20098 raft_consensus.cc:1261] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Refusing update from remote peer 5beed0f7b84d40da9270b2cea8842804: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:32.936811 20213 consensus_queue.cc:1035] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:32.937012 20247 consensus_queue.cc:1035] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:32.948794 19606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:32.948920 19606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:32.949693 19606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: Sending DeleteTablet for 3 replicas of tablet befa592cc457488184429de5d3f444ef
I20260812 03:52:32.949980 19762 tablet_service.cc:1514] Processing DeleteTablet for tablet befa592cc457488184429de5d3f444ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:32 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:44154
I20260812 03:52:32.949990 19919 tablet_service.cc:1514] Processing DeleteTablet for tablet befa592cc457488184429de5d3f444ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:32 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:45156
I20260812 03:52:32.950021 20064 tablet_service.cc:1514] Processing DeleteTablet for tablet befa592cc457488184429de5d3f444ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:32 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:60760
I20260812 03:52:32.950080 20267 tablet_replica.cc:330] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: stopping tablet replica
I20260812 03:52:32.950080 20268 tablet_replica.cc:330] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: stopping tablet replica
I20260812 03:52:32.950115 20270 tablet_replica.cc:330] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: stopping tablet replica
I20260812 03:52:32.950133 20267 raft_consensus.cc:2229] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:32.950136 20268 raft_consensus.cc:2229] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:32.950161 20270 raft_consensus.cc:2229] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:32.950173 20268 raft_consensus.cc:2258] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:32.950201 20270 raft_consensus.cc:2258] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:32.950220 20267 raft_consensus.cc:2258] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:32.950675 20267 ts_tablet_manager.cc:1905] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:32.950704 20270 ts_tablet_manager.cc:1905] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:32.950780 20268 ts_tablet_manager.cc:1905] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:32.950747 19606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:32.950909 19606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:32.952353 20064 tablet_service.cc:1467] Processing CreateTablet for tablet 346503a044154b0289fd5936109ddfef (DEFAULT_TABLE table=test-table [id=090b81ef132344339d71c6a04feee4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:32.952498 19762 tablet_service.cc:1467] Processing CreateTablet for tablet 346503a044154b0289fd5936109ddfef (DEFAULT_TABLE table=test-table [id=090b81ef132344339d71c6a04feee4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:32.952458 19919 tablet_service.cc:1467] Processing CreateTablet for tablet 346503a044154b0289fd5936109ddfef (DEFAULT_TABLE table=test-table [id=090b81ef132344339d71c6a04feee4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:32.952602 20267 ts_tablet_manager.cc:1918] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:32.952607 20270 ts_tablet_manager.cc:1918] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:32.952630 19919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346503a044154b0289fd5936109ddfef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:32.952637 20268 ts_tablet_manager.cc:1918] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:32.952657 20270 log.cc:1196] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/befa592cc457488184429de5d3f444ef
I20260812 03:52:32.952657 20267 log.cc:1196] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/befa592cc457488184429de5d3f444ef
I20260812 03:52:32.952682 20268 log.cc:1196] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/befa592cc457488184429de5d3f444ef
I20260812 03:52:32.952899 20270 ts_tablet_manager.cc:1939] T befa592cc457488184429de5d3f444ef P f2bc4088aad94fa3b331075ab65e6b61: Deleting consensus metadata
I20260812 03:52:32.952908 20268 ts_tablet_manager.cc:1939] T befa592cc457488184429de5d3f444ef P 04ad518a9ffb4452b0a9785b4e8ae419: Deleting consensus metadata
I20260812 03:52:32.952899 20267 ts_tablet_manager.cc:1939] T befa592cc457488184429de5d3f444ef P 5beed0f7b84d40da9270b2cea8842804: Deleting consensus metadata
I20260812 03:52:32.953044 19762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346503a044154b0289fd5936109ddfef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:32.953049 20064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346503a044154b0289fd5936109ddfef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:32.953963 20281 tablet_bootstrap.cc:492] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: Bootstrap starting.
I20260812 03:52:32.954243 20279 tablet_bootstrap.cc:492] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: Bootstrap starting.
I20260812 03:52:32.954279 20281 tablet_bootstrap.cc:654] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:32.954653 20279 tablet_bootstrap.cc:654] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:32.954648 19590 catalog_manager.cc:4841] TS f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881): tablet befa592cc457488184429de5d3f444ef (table test-table [id=919eac0c7a184b12ad530c43ab503769]) successfully deleted
I20260812 03:52:32.954869 19590 catalog_manager.cc:4841] TS 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037): tablet befa592cc457488184429de5d3f444ef (table test-table [id=919eac0c7a184b12ad530c43ab503769]) successfully deleted
I20260812 03:52:32.954967 20280 tablet_bootstrap.cc:492] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419: Bootstrap starting.
I20260812 03:52:32.955236 20281 tablet_bootstrap.cc:492] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: No bootstrap required, opened a new log
I20260812 03:52:32.955286 20281 ts_tablet_manager.cc:1397] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:32.955298 20280 tablet_bootstrap.cc:654] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:32.955387 20281 raft_consensus.cc:348] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.955451 20281 raft_consensus.cc:374] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:32.955472 20281 raft_consensus.cc:729] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2bc4088aad94fa3b331075ab65e6b61, State: Initialized, Role: FOLLOWER
I20260812 03:52:32.955516 20281 consensus_queue.cc:260] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.955605 20281 ts_tablet_manager.cc:1428] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:32.955838 19592 catalog_manager.cc:4841] TS 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099): tablet befa592cc457488184429de5d3f444ef (table test-table [id=919eac0c7a184b12ad530c43ab503769]) successfully deleted
I20260812 03:52:32.956100 20280 tablet_bootstrap.cc:492] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419: No bootstrap required, opened a new log
I20260812 03:52:32.956152 20280 ts_tablet_manager.cc:1397] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:32.956260 20280 raft_consensus.cc:348] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.956331 20280 raft_consensus.cc:374] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:32.956353 20280 raft_consensus.cc:729] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ad518a9ffb4452b0a9785b4e8ae419, State: Initialized, Role: FOLLOWER
I20260812 03:52:32.956408 20280 consensus_queue.cc:260] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.956501 20280 ts_tablet_manager.cc:1428] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:32.957036 20279 tablet_bootstrap.cc:492] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: No bootstrap required, opened a new log
I20260812 03:52:32.957100 20279 ts_tablet_manager.cc:1397] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:52:32.957250 20279 raft_consensus.cc:348] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.957321 20279 raft_consensus.cc:374] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:32.957340 20279 raft_consensus.cc:729] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5beed0f7b84d40da9270b2cea8842804, State: Initialized, Role: FOLLOWER
I20260812 03:52:32.957402 20279 consensus_queue.cc:260] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:32.957497 20279 ts_tablet_manager.cc:1428] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:33.033093 20247 raft_consensus.cc:482] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:33.033202 20247 raft_consensus.cc:504] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:33.033401 20247 leader_election.cc:288] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099), f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:33.033670 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346503a044154b0289fd5936109ddfef" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2bc4088aad94fa3b331075ab65e6b61" is_pre_election: true
I20260812 03:52:33.033670 19784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346503a044154b0289fd5936109ddfef" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" is_pre_election: true
I20260812 03:52:33.033785 20099 raft_consensus.cc:2454] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 0.
I20260812 03:52:33.033793 19784 raft_consensus.cc:2454] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 0.
I20260812 03:52:33.034011 19869 leader_election.cc:302] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ad518a9ffb4452b0a9785b4e8ae419, 5beed0f7b84d40da9270b2cea8842804; no voters:
I20260812 03:52:33.034147 20247 raft_consensus.cc:2781] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:33.034189 20247 raft_consensus.cc:482] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:33.034216 20247 raft_consensus.cc:3037] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:33.034997 20247 raft_consensus.cc:504] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:33.035176 20247 leader_election.cc:288] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 election: Requested vote from peers 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099), f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881)
I20260812 03:52:33.035375 19784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346503a044154b0289fd5936109ddfef" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04ad518a9ffb4452b0a9785b4e8ae419"
I20260812 03:52:33.035471 19784 raft_consensus.cc:3037] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:33.035478 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346503a044154b0289fd5936109ddfef" candidate_uuid: "5beed0f7b84d40da9270b2cea8842804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2bc4088aad94fa3b331075ab65e6b61"
I20260812 03:52:33.035583 20099 raft_consensus.cc:3037] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:33.036302 20099 raft_consensus.cc:2454] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 1.
I20260812 03:52:33.036386 19784 raft_consensus.cc:2454] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5beed0f7b84d40da9270b2cea8842804 in term 1.
I20260812 03:52:33.036520 19866 leader_election.cc:302] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5beed0f7b84d40da9270b2cea8842804, f2bc4088aad94fa3b331075ab65e6b61; no voters:
I20260812 03:52:33.036692 20247 raft_consensus.cc:2781] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:33.036767 20247 raft_consensus.cc:686] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 1 LEADER]: Becoming Leader. State: Replica: 5beed0f7b84d40da9270b2cea8842804, State: Running, Role: LEADER
I20260812 03:52:33.036837 20247 consensus_queue.cc:237] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } }
I20260812 03:52:33.037590 19607 catalog_manager.cc:5495] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194). New cstate: current_term: 1 leader_uuid: "5beed0f7b84d40da9270b2cea8842804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5beed0f7b84d40da9270b2cea8842804" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 38037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:33.055047 20099 raft_consensus.cc:1261] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Refusing update from remote peer 5beed0f7b84d40da9270b2cea8842804: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:33.055140 19784 raft_consensus.cc:1261] T 346503a044154b0289fd5936109ddfef P 04ad518a9ffb4452b0a9785b4e8ae419 [term 1 FOLLOWER]: Refusing update from remote peer 5beed0f7b84d40da9270b2cea8842804: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:33.055351 20247 consensus_queue.cc:1035] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04ad518a9ffb4452b0a9785b4e8ae419" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:33.055521 20252 consensus_queue.cc:1035] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2bc4088aad94fa3b331075ab65e6b61" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:52:33.061796 20245 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:52:34.580992 19606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:34.581100 19606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:34.582073 19606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 371a132e40be4183931295c089766fd0: Sending DeleteTablet for 3 replicas of tablet 346503a044154b0289fd5936109ddfef
I20260812 03:52:34.582413 19904 tablet_service.cc:1514] Processing DeleteTablet for tablet 346503a044154b0289fd5936109ddfef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:34 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:45156
I20260812 03:52:34.582449 19762 tablet_service.cc:1514] Processing DeleteTablet for tablet 346503a044154b0289fd5936109ddfef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:34 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:44154
I20260812 03:52:34.582446 20064 tablet_service.cc:1514] Processing DeleteTablet for tablet 346503a044154b0289fd5936109ddfef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:34 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:60760
I20260812 03:52:34.582638 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 19686
I20260812 03:52:34.582659 20330 tablet_replica.cc:330] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: stopping tablet replica
I20260812 03:52:34.582718 20330 raft_consensus.cc:2229] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:34.583894 20330 raft_consensus.cc:2258] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:34.591229 20330 ts_tablet_manager.cc:1905] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:34.593294 20330 ts_tablet_manager.cc:1918] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:52:34.593357 20330 log.cc:1196] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/346503a044154b0289fd5936109ddfef
I20260812 03:52:34.593643 20330 ts_tablet_manager.cc:1939] T 346503a044154b0289fd5936109ddfef P 5beed0f7b84d40da9270b2cea8842804: Deleting consensus metadata
I20260812 03:52:34.595432 20331 tablet_replica.cc:330] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: stopping tablet replica
I20260812 03:52:34.595535 20331 raft_consensus.cc:2229] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:34.595610 20331 raft_consensus.cc:2258] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:34.595942 20331 ts_tablet_manager.cc:1905] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:34.596237 19590 catalog_manager.cc:4841] TS 5beed0f7b84d40da9270b2cea8842804 (127.17.54.194:38037): tablet 346503a044154b0289fd5936109ddfef (table test-table [id=090b81ef132344339d71c6a04feee4bd]) successfully deleted
I20260812 03:52:34.597916 20331 ts_tablet_manager.cc:1918] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:52:34.597997 20331 log.cc:1196] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/346503a044154b0289fd5936109ddfef
I20260812 03:52:34.598235 20331 ts_tablet_manager.cc:1939] T 346503a044154b0289fd5936109ddfef P f2bc4088aad94fa3b331075ab65e6b61: Deleting consensus metadata
I20260812 03:52:34.598816 19590 catalog_manager.cc:4841] TS f2bc4088aad94fa3b331075ab65e6b61 (127.17.54.195:45881): tablet 346503a044154b0289fd5936109ddfef (table test-table [id=090b81ef132344339d71c6a04feee4bd]) successfully deleted
W20260812 03:52:34.601637 19592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:33099 (error 108)
I20260812 03:52:34.601884 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 19847
I20260812 03:52:34.603905 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:34.602022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33099 (local address 127.0.0.1:44160)
0812 03:52:34.603773 (+ 1751us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:34.603847 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1715,"thread_start_us":59,"threads_started":1}
W20260812 03:52:34.605509 19592 catalog_manager.cc:4584] TS 04ad518a9ffb4452b0a9785b4e8ae419 (127.17.54.193:33099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 346503a044154b0289fd5936109ddfef: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33099: connect: Connection refused (error 111)
I20260812 03:52:34.611065 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 20000
I20260812 03:52:34.618783 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 19574
2026-08-12T03:52:34Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6110 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:34 dist-test-slave-tm1g krb5kdc[20337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:34 dist-test-slave-tm1g krb5kdc[20337](info): set up 2 sockets
Aug 12 03:52:34 dist-test-slave-tm1g krb5kdc[20337](info): commencing operation
krb5kdc: starting...
W20260812 03:52:35.771445 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:35 dist-test-slave-tm1g krb5kdc[20337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:35.902542 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37409
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:37409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:35.990787 20351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:35.991029 20351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:35.991074 20351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:35.992259 20351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:35.992336 20351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:35.992364 20351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:35.992395 20351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:35.992420 20351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:35.993945 20351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:37409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:35.994227 20351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:35.994500 20351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:52:36.000165 20351 server_base.cc:1034] running on GCE node
W20260812 03:52:36.000149 20360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:36.000149 20356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:36.000167 20358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:36.000653 20351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:36.000957 20351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:36.002148 20351 hybrid_clock.cc:648] HybridClock initialized: now 1786506756002130 us; error 49 us; skew 500 ppm
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506756, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:36.010835 20351 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:36.011368 20351 webserver.cc:460] Webserver started at http://127.17.54.254:45397/ using document root <none> and password file <none>
I20260812 03:52:36.011644 20351 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:36.011693 20351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:36.011852 20351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:36.012969 20351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "39bb31c7368541238bef98dbee05a603"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "f6c2f2f9946fa9d8fba83e7da9bbc77b"
server_key_iv: "692e09121351a425948191b33082075e"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.013361 20351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "39bb31c7368541238bef98dbee05a603"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "f6c2f2f9946fa9d8fba83e7da9bbc77b"
server_key_iv: "692e09121351a425948191b33082075e"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.014859 20351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:36.015726 20370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.015946 20351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:36.016047 20351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "39bb31c7368541238bef98dbee05a603"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "f6c2f2f9946fa9d8fba83e7da9bbc77b"
server_key_iv: "692e09121351a425948191b33082075e"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.016108 20351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:36.038079 20351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:36.040546 20351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:36.040673 20351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:36.044859 20351 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:37409
I20260812 03:52:36.044919 20429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:37409 every 8 connection(s)
I20260812 03:52:36.045306 20351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:36.046561 20430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:36.048733 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 20351
I20260812 03:52:36.048815 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:36.049006 17627 external_mini_cluster.cc:1428] Setting key dce8d8d3be4583f2d18214578391ed51
I20260812 03:52:36.049062 20430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603: Bootstrap starting.
I20260812 03:52:36.049886 20430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:36.050222 20430 log.cc:824] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:36.051038 20430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603: No bootstrap required, opened a new log
I20260812 03:52:36.053521 20430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bb31c7368541238bef98dbee05a603" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37409 } }
I20260812 03:52:36.053711 20430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:36.053748 20430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39bb31c7368541238bef98dbee05a603, State: Initialized, Role: FOLLOWER
I20260812 03:52:36.053885 20430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bb31c7368541238bef98dbee05a603" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37409 } }
I20260812 03:52:36.053959 20430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:36.053989 20430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:36.054034 20430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:36.054775 20430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bb31c7368541238bef98dbee05a603" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37409 } }
I20260812 03:52:36.054915 20430 leader_election.cc:302] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39bb31c7368541238bef98dbee05a603; no voters:
I20260812 03:52:36.055137 20430 leader_election.cc:288] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:36.055178 20436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:36.055315 20436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [term 1 LEADER]: Becoming Leader. State: Replica: 39bb31c7368541238bef98dbee05a603, State: Running, Role: LEADER
I20260812 03:52:36.055425 20430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:36.055402 20436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bb31c7368541238bef98dbee05a603" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37409 } }
I20260812 03:52:36.055729 20438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39bb31c7368541238bef98dbee05a603. Latest consensus state: current_term: 1 leader_uuid: "39bb31c7368541238bef98dbee05a603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bb31c7368541238bef98dbee05a603" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37409 } } }
I20260812 03:52:36.055809 20438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:36.055752 20437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39bb31c7368541238bef98dbee05a603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bb31c7368541238bef98dbee05a603" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37409 } } }
I20260812 03:52:36.055990 20437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:36.056565 20440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:36.057085 20440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:36.058777 20440 catalog_manager.cc:1347] Generated new cluster ID: ae59a8d20e564c7084f555b79081520d
I20260812 03:52:36.058833 20440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:36.061723 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:36.049593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.17.54.254:37409)
0812 03:52:36.049820 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:36.049830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:36.049869 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:52:36.050527 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:36.050534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:36.050583 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:36.050739 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:36.051390 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.052871 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.053647 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.054283 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.054439 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:36.057492 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:36.057497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:36.057497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:36.057510 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:36.059628 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:36.060100 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:36.060104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:36.060104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:36.060207 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:36.060417 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:36.060418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:36.060418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:36.060721 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:36.060851 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:36.061336 (+ 485us) server_negotiation.cc:299] Negotiation successful
0812 03:52:36.061498 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
I20260812 03:52:36.062983 20440 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:36.064059 20440 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:36.072074 20440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39bb31c7368541238bef98dbee05a603: Generated new TSK 0
I20260812 03:52:36.072259 20440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:36.102540 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37409
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:36.189903 20459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:36.190143 20459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:36.190186 20459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:36.191459 20459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:36.191530 20459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:36.191576 20459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:36.193153 20459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:36.193418 20459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:36.193655 20459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:36.198976 20470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:36.199002 20468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:36.199141 20459 server_base.cc:1034] running on GCE node
W20260812 03:52:36.199003 20466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:36.199410 20459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:36.199679 20459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:36.200840 20459 hybrid_clock.cc:648] HybridClock initialized: now 1786506756200841 us; error 41 us; skew 500 ppm
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506756, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:36.203358 20459 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:36.203868 20459 webserver.cc:460] Webserver started at http://127.17.54.193:33665/ using document root <none> and password file <none>
I20260812 03:52:36.204106 20459 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:36.204174 20459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:36.204329 20459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:36.205435 20459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "a60bd61d6b8d425a988a7b0a43c1d641"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "e22ec89b357f95baafd2fb0461ab62a6"
server_key_iv: "83959cbeef16762446e64bf5a2d1f022"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.205809 20459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "a60bd61d6b8d425a988a7b0a43c1d641"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "e22ec89b357f95baafd2fb0461ab62a6"
server_key_iv: "83959cbeef16762446e64bf5a2d1f022"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.207244 20459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:36.208015 20477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.208191 20459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.208299 20459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "a60bd61d6b8d425a988a7b0a43c1d641"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "e22ec89b357f95baafd2fb0461ab62a6"
server_key_iv: "83959cbeef16762446e64bf5a2d1f022"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.208374 20459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:36.234285 20459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:36.236382 20459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:36.236562 20459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:36.236821 20459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:36.237155 20459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:36.237212 20459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.237258 20459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:36.237284 20459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.243640 20459 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:42323
I20260812 03:52:36.243700 20601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:42323 every 8 connection(s)
I20260812 03:52:36.244028 20459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:36.249231 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 20459
I20260812 03:52:36.249320 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:36.249523 17627 external_mini_cluster.cc:1428] Setting key c804e2b11f55bf9085f8d12e4b81488c
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506756, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:36.254263 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:36.244771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:48529 (local address 127.17.54.254:37409)
0812 03:52:36.244822 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:36.244825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:36.245024 (+ 199us) server_negotiation.cc:407] Connection header received
0812 03:52:36.245532 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:36.245533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:36.245544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:36.245609 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:36.246676 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.247209 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.248198 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.248403 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.248460 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:36.250869 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:36.250870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:36.250871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:36.250877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:36.252584 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:36.253212 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:36.253214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:36.253214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:36.253272 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:36.253615 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:36.253615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:36.253616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:36.253724 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:36.253777 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:36.254123 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 03:52:36.254199 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:52:36.254609 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:36.244716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37409 (local address 127.17.54.193:48529)
0812 03:52:36.244883 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:36.244927 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:36.245444 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:36.245683 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:36.245687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:36.246261 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:36.246586 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:36.246591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.247295 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.247296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:36.248120 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:36.248123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.248483 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.248484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:36.248588 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:36.249079 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:36.249087 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:36.250762 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:36.252736 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:36.252746 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:36.252748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:36.253128 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:36.253345 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:36.253346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:36.253347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:36.253546 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:36.253793 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:36.253800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:36.253982 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 03:52:36.254105 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 03:52:36.254289 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260812 03:52:36.255215 20602 heartbeater.cc:347] Connected to a master server at 127.17.54.254:37409
I20260812 03:52:36.255338 20602 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:36.255537 20602 heartbeater.cc:511] Master 127.17.54.254:37409 requested a full tablet report, sending...
I20260812 03:52:36.256151 20390 ts_manager.cc:194] Registered new tserver with Master: a60bd61d6b8d425a988a7b0a43c1d641 (127.17.54.193:42323)
I20260812 03:52:36.256847 20390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:48529
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:36.279598 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37409
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:36.365728 20609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:36.365928 20609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:36.365952 20609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:36.367164 20609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:36.367206 20609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:36.367233 20609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:36.368711 20609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:36.368970 20609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:36.369220 20609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:36.374764 20618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:36.374752 20616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:36.374933 20609 server_base.cc:1034] running on GCE node
W20260812 03:52:36.374754 20615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:36.375243 20609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:36.375512 20609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:36.376662 20609 hybrid_clock.cc:648] HybridClock initialized: now 1786506756376650 us; error 30 us; skew 500 ppm
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506756, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:36.379310 20609 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:36.379812 20609 webserver.cc:460] Webserver started at http://127.17.54.194:39781/ using document root <none> and password file <none>
I20260812 03:52:36.380069 20609 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:36.380137 20609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:36.380285 20609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:36.381340 20609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "d9461fd270df40f88b756c9c63d4a77d"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "2a5725d121799d0fc4a0d3b0c4199dfa"
server_key_iv: "9afcfceca35b62ed6d2d121b7bf184ce"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.381706 20609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "d9461fd270df40f88b756c9c63d4a77d"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "2a5725d121799d0fc4a0d3b0c4199dfa"
server_key_iv: "9afcfceca35b62ed6d2d121b7bf184ce"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.383167 20609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:36.383985 20625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.384186 20609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:36.384291 20609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "d9461fd270df40f88b756c9c63d4a77d"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "2a5725d121799d0fc4a0d3b0c4199dfa"
server_key_iv: "9afcfceca35b62ed6d2d121b7bf184ce"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.384366 20609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:36.408581 20609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:36.411062 20609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:36.411224 20609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:36.411475 20609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:36.411856 20609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:36.411929 20609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.411963 20609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:36.412052 20609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.418232 20609 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:37413
I20260812 03:52:36.418357 20748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:37413 every 8 connection(s)
I20260812 03:52:36.418691 20609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506756, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:36.425824 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 20609
I20260812 03:52:36.425938 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:36.426177 17627 external_mini_cluster.cc:1428] Setting key 007d0ffb0b53b725ee8af99aee33b7d0
I20260812 03:52:36.428725 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:36.419481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:54027 (local address 127.17.54.254:37409)
0812 03:52:36.419542 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:36.419544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:36.419756 (+ 212us) server_negotiation.cc:407] Connection header received
0812 03:52:36.420320 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:36.420322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:36.420335 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:36.420384 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:36.421400 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.421941 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.423034 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.423238 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.423313 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:36.425506 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:36.425507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:36.425508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:36.425515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:36.427320 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:36.427737 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:36.427739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:36.427739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:36.427802 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:36.428038 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:36.428039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:36.428039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:36.428185 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:36.428253 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:36.428503 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 03:52:36.428591 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:52:36.429001 20751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:36.419455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37409 (local address 127.17.54.194:54027)
0812 03:52:36.419629 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:36.419669 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:36.420199 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:36.420460 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:36.420464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:36.421077 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:36.421276 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:36.421281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.422041 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.422042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:36.422937 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:36.422940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.423333 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.423334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:36.423470 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:36.423925 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:36.423934 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:36.425342 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:36.427404 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:36.427411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:36.427413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:36.427659 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:36.427870 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:36.427871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:36.427871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:36.427974 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:36.428246 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:36.428251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:36.428365 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 03:52:36.428469 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:52:36.428594 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260812 03:52:36.429438 20749 heartbeater.cc:347] Connected to a master server at 127.17.54.254:37409
I20260812 03:52:36.429534 20749 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:36.429749 20749 heartbeater.cc:511] Master 127.17.54.254:37409 requested a full tablet report, sending...
I20260812 03:52:36.430171 20390 ts_manager.cc:194] Registered new tserver with Master: d9461fd270df40f88b756c9c63d4a77d (127.17.54.194:37413)
I20260812 03:52:36.430709 20390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:54027
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:36.453749 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37409
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:36.538424 20754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:36.538630 20754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:36.538654 20754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:36.539845 20754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:36.539887 20754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:36.539915 20754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:36.541396 20754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:36.541643 20754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:36.541877 20754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:36.547026 20762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:36.547081 20765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:36.547084 20761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:36.547286 20754 server_base.cc:1034] running on GCE node
I20260812 03:52:36.547592 20754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:36.547842 20754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:36.549016 20754 hybrid_clock.cc:648] HybridClock initialized: now 1786506756549005 us; error 38 us; skew 500 ppm
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506756, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:36.551522 20754 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:36.551985 20754 webserver.cc:460] Webserver started at http://127.17.54.195:38153/ using document root <none> and password file <none>
I20260812 03:52:36.552181 20754 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:36.552220 20754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:36.552309 20754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:36.553314 20754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "d5002d704a8e42afab4c45cb01fdaaae"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "ad388323ae5d3827036291f753be2e0b"
server_key_iv: "ce084e542ed24cc809f4d89a4ee6bbb0"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.553617 20754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "d5002d704a8e42afab4c45cb01fdaaae"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "ad388323ae5d3827036291f753be2e0b"
server_key_iv: "ce084e542ed24cc809f4d89a4ee6bbb0"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.554970 20754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:36.555735 20772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.555951 20754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:36.556046 20754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "d5002d704a8e42afab4c45cb01fdaaae"
format_stamp: "Formatted at 2026-08-12 03:52:36 on dist-test-slave-tm1g"
server_key: "ad388323ae5d3827036291f753be2e0b"
server_key_iv: "ce084e542ed24cc809f4d89a4ee6bbb0"
server_key_version: "encryptionkey@0"
I20260812 03:52:36.556140 20754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:36.577356 20754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:36.579828 20754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:36.579985 20754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:36.580281 20754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:36.580631 20754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:36.580688 20754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.580724 20754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:36.580771 20754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:36.586867 20754 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:44103
I20260812 03:52:36.586915 20898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:44103 every 8 connection(s)
I20260812 03:52:36.587275 20754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:36.589727 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 20754
I20260812 03:52:36.589819 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:36.590025 17627 external_mini_cluster.cc:1428] Setting key 8712a9098477120d2948bbdd79940421
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506756, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:36.597112 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:36.588023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:38719 (local address 127.17.54.254:37409)
0812 03:52:36.588073 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:36.588075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:36.588279 (+ 204us) server_negotiation.cc:407] Connection header received
0812 03:52:36.588869 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:36.588870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:36.588879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:36.588939 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:36.590114 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.590962 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.591913 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.592098 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.592148 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:36.594354 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:36.594356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:36.594356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:36.594364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:36.595832 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:36.596220 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:36.596221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:36.596221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:36.596283 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:36.596559 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:36.596559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:36.596560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:36.596670 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:36.596723 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:36.596981 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:52:36.597047 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:52:36.597302 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:36.587982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37409 (local address 127.17.54.195:38719)
0812 03:52:36.588157 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:36.588203 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:36.588773 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:36.588991 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:36.588995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:36.589671 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:36.589994 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:36.590001 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.591036 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.591037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:36.591840 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:36.591842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:36.592169 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:36.592170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:36.592270 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:36.592744 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:36.592753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:36.594224 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:36.595906 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:36.595912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:36.595914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:36.596130 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:36.596382 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:36.596383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:36.596384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:36.596489 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:36.596737 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:36.596743 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:36.596866 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 03:52:36.596956 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:52:36.597085 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260812 03:52:36.597832 20899 heartbeater.cc:347] Connected to a master server at 127.17.54.254:37409
I20260812 03:52:36.597929 20899 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:36.598129 20899 heartbeater.cc:511] Master 127.17.54.254:37409 requested a full tablet report, sending...
I20260812 03:52:36.598555 20390 ts_manager.cc:194] Registered new tserver with Master: d5002d704a8e42afab4c45cb01fdaaae (127.17.54.195:44103)
I20260812 03:52:36.598922 20390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:38719
I20260812 03:52:36.601897 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:52:36.614714 20434 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:36.613652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.17.54.254:37409)
0812 03:52:36.613730 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:36.613732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:36.613880 (+ 148us) server_negotiation.cc:407] Connection header received
0812 03:52:36.613914 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:36.613915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:36.613926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:36.613992 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:36.614655 (+ 663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44648: BlockingRecv error: recv got EOF from 127.0.0.1:44648 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:52:36.615607 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 20459
I20260812 03:52:36.624714 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 20609
I20260812 03:52:36.632409 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 20754
I20260812 03:52:36.639554 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 20351
2026-08-12T03:52:36Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2020 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20913](info): set up 2 sockets
Aug 12 03:52:36 dist-test-slave-tm1g krb5kdc[20913](info): commencing operation
krb5kdc: starting...
W20260812 03:52:37.756366 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:37 dist-test-slave-tm1g krb5kdc[20913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:37.885562 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37199
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:35897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:37199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:37.969092 20930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:37.969331 20930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:37.969369 20930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:37.970633 20930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:37.970695 20930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:37.970718 20930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:37.970741 20930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:37.970772 20930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:37.972323 20930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:35897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:37199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:37.972616 20930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:37.972851 20930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:52:37.977934 20930 server_base.cc:1034] running on GCE node
W20260812 03:52:37.977927 20935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:37.977921 20939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:37.977955 20936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:37.978446 20930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:37.978747 20930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:37.979902 20930 hybrid_clock.cc:648] HybridClock initialized: now 1786506757979878 us; error 38 us; skew 500 ppm
Aug 12 03:52:37 dist-test-slave-tm1g krb5kdc[20913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506757, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:37.982599 20930 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:37.983057 20930 webserver.cc:460] Webserver started at http://127.17.54.254:45931/ using document root <none> and password file <none>
I20260812 03:52:37.983305 20930 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:37.983376 20930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:37.983523 20930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:37.984606 20930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd"
format_stamp: "Formatted at 2026-08-12 03:52:37 on dist-test-slave-tm1g"
server_key: "6c80f8de8c90616e77bce3c6e6b3726a"
server_key_iv: "443958ba93b7a6666c52f697e922266c"
server_key_version: "encryptionkey@0"
I20260812 03:52:37.984988 20930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd"
format_stamp: "Formatted at 2026-08-12 03:52:37 on dist-test-slave-tm1g"
server_key: "6c80f8de8c90616e77bce3c6e6b3726a"
server_key_iv: "443958ba93b7a6666c52f697e922266c"
server_key_version: "encryptionkey@0"
I20260812 03:52:37.986451 20930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:37.987237 20946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:37.987414 20930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:37.987527 20930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd"
format_stamp: "Formatted at 2026-08-12 03:52:37 on dist-test-slave-tm1g"
server_key: "6c80f8de8c90616e77bce3c6e6b3726a"
server_key_iv: "443958ba93b7a6666c52f697e922266c"
server_key_version: "encryptionkey@0"
I20260812 03:52:37.987603 20930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:37.999751 20930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:38.002041 20930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:38.002209 20930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:38.006634 20930 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:37199
I20260812 03:52:38.006683 21002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:37199 every 8 connection(s)
I20260812 03:52:38.007270 20930 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:38.007988 21003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:38.010452 21003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd: Bootstrap starting.
I20260812 03:52:38.011081 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 20930
I20260812 03:52:38.011188 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:38.011204 21003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:38.011411 17627 external_mini_cluster.cc:1428] Setting key 46aad2f4a6ba4b445d96c9eccc995840
I20260812 03:52:38.011580 21003 log.cc:824] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:38.012574 21003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd: No bootstrap required, opened a new log
I20260812 03:52:38.014581 21003 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37199 } }
I20260812 03:52:38.014744 21003 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:38.014794 21003 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8422e99ecfe4bfdb64190d06ecd5fcd, State: Initialized, Role: FOLLOWER
I20260812 03:52:38.014925 21003 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37199 } }
I20260812 03:52:38.014986 21003 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:38.015007 21003 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:38.015061 21003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:38.015646 21003 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37199 } }
I20260812 03:52:38.015738 21003 leader_election.cc:302] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8422e99ecfe4bfdb64190d06ecd5fcd; no voters:
I20260812 03:52:38.015969 21003 leader_election.cc:288] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:38.015996 21010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:38.016115 21010 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [term 1 LEADER]: Becoming Leader. State: Replica: d8422e99ecfe4bfdb64190d06ecd5fcd, State: Running, Role: LEADER
I20260812 03:52:38.016202 21010 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37199 } }
I20260812 03:52:38.016253 21003 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:38.016530 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37199 } } }
I20260812 03:52:38.016603 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:38.016743 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8422e99ecfe4bfdb64190d06ecd5fcd. Latest consensus state: current_term: 1 leader_uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8422e99ecfe4bfdb64190d06ecd5fcd" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37199 } } }
I20260812 03:52:38.016819 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:38.017452 21015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:38.017971 21015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:52:38 dist-test-slave-tm1g krb5kdc[20913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:38.019670 21015 catalog_manager.cc:1347] Generated new cluster ID: 5a3957cb2d8f4167afc64b41ea378cdd
I20260812 03:52:38.019724 21015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:38.022832 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:38.012054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.17.54.254:37199)
0812 03:52:38.012229 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:38.012238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:38.012273 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:52:38.012776 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:38.012783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:38.012859 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:38.013014 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:38.013715 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:38.014661 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:38.015582 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:38.016148 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:38.016287 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:38.019397 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:38.019401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:38.019401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:38.019414 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:38.021245 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:38.021615 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:38.021618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:38.021618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:38.021710 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:38.021922 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:38.021923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:38.021923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:38.022162 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:38.022368 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:38.022444 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:52:38.022592 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260812 03:52:38.023698 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:38.032624 21015 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:38.033349 21015 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:38.039117 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:38.035551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.17.54.254:37199)
0812 03:52:38.035595 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:38.035597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:38.035730 (+ 133us) server_negotiation.cc:407] Connection header received
0812 03:52:38.035804 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:38.035805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:38.035816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:38.035878 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:38.036283 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:38.036791 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:38.037547 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:38.037782 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:38.037867 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:38.038623 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:38.038624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:38.038624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:38.038820 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:38.038844 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:38.038850 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:38.038919 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:38.038984 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:52:38.039014 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:52:38.040807 21015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8422e99ecfe4bfdb64190d06ecd5fcd: Generated new TSK 0
I20260812 03:52:38.041010 21015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:38.046465 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 20930
2026-08-12T03:52:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1407 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:38 dist-test-slave-tm1g krb5kdc[21040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:38 dist-test-slave-tm1g krb5kdc[21040](info): set up 2 sockets
Aug 12 03:52:38 dist-test-slave-tm1g krb5kdc[21040](info): commencing operation
krb5kdc: starting...
W20260812 03:52:39.170616 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:39 dist-test-slave-tm1g krb5kdc[21040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:39.305729 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:35391
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:35391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:39.390489 21054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:39.390705 21054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:39.390729 21054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:39.392007 21054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:39.392055 21054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:39.392076 21054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:39.392091 21054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:39.392103 21054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:39.393651 21054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:35391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:35391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:39.393954 21054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:39.394204 21054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:39.399675 21060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:39.399693 21059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:39.399760 21054 server_base.cc:1034] running on GCE node
W20260812 03:52:39.399686 21063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:39.400082 21054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:39.400396 21054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:39.401614 21054 hybrid_clock.cc:648] HybridClock initialized: now 1786506759401590 us; error 36 us; skew 500 ppm
Aug 12 03:52:39 dist-test-slave-tm1g krb5kdc[21040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:39.404335 21054 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:39.404884 21054 webserver.cc:460] Webserver started at http://127.17.54.254:36007/ using document root <none> and password file <none>
I20260812 03:52:39.405123 21054 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:39.405194 21054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:39.405356 21054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:39.406543 21054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "a05fa836b510440d99c85af1e7317345"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "e94f4d7c103107a957ab4f3e1d827286"
server_key_iv: "23c4d5ff6612ba503ef243565265b4b6"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.406934 21054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "a05fa836b510440d99c85af1e7317345"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "e94f4d7c103107a957ab4f3e1d827286"
server_key_iv: "23c4d5ff6612ba503ef243565265b4b6"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.408424 21054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:39.409314 21071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.409528 21054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:39.409641 21054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "a05fa836b510440d99c85af1e7317345"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "e94f4d7c103107a957ab4f3e1d827286"
server_key_iv: "23c4d5ff6612ba503ef243565265b4b6"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.409722 21054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:39.420276 21054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:39.422729 21054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:39.422899 21054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:39.427330 21054 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:35391
I20260812 03:52:39.427374 21131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:35391 every 8 connection(s)
I20260812 03:52:39.427772 21054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:39.428992 21133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:39.431458 21133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345: Bootstrap starting.
I20260812 03:52:39.431736 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 21054
I20260812 03:52:39.431856 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:39.432072 17627 external_mini_cluster.cc:1428] Setting key c36567563a1b2d837d81651437a858ac
I20260812 03:52:39.432209 21133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:39.432586 21133 log.cc:824] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:39.433373 21133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345: No bootstrap required, opened a new log
I20260812 03:52:39.435362 21133 raft_consensus.cc:348] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05fa836b510440d99c85af1e7317345" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35391 } }
I20260812 03:52:39.435529 21133 raft_consensus.cc:374] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:39.435576 21133 raft_consensus.cc:729] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a05fa836b510440d99c85af1e7317345, State: Initialized, Role: FOLLOWER
I20260812 03:52:39.435753 21133 consensus_queue.cc:260] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05fa836b510440d99c85af1e7317345" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35391 } }
I20260812 03:52:39.435870 21133 raft_consensus.cc:388] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:39.435917 21133 raft_consensus.cc:482] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:39.435968 21133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:39.436556 21133 raft_consensus.cc:504] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05fa836b510440d99c85af1e7317345" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35391 } }
I20260812 03:52:39.436683 21133 leader_election.cc:302] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a05fa836b510440d99c85af1e7317345; no voters:
I20260812 03:52:39.436892 21133 leader_election.cc:288] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:39.436919 21138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:39.437115 21138 raft_consensus.cc:686] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [term 1 LEADER]: Becoming Leader. State: Replica: a05fa836b510440d99c85af1e7317345, State: Running, Role: LEADER
I20260812 03:52:39.437248 21133 sys_catalog.cc:564] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:39.437252 21138 consensus_queue.cc:237] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05fa836b510440d99c85af1e7317345" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35391 } }
I20260812 03:52:39.437711 21140 sys_catalog.cc:455] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a05fa836b510440d99c85af1e7317345. Latest consensus state: current_term: 1 leader_uuid: "a05fa836b510440d99c85af1e7317345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05fa836b510440d99c85af1e7317345" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35391 } } }
I20260812 03:52:39.437886 21140 sys_catalog.cc:458] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:39.437711 21139 sys_catalog.cc:455] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a05fa836b510440d99c85af1e7317345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05fa836b510440d99c85af1e7317345" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35391 } } }
I20260812 03:52:39.438553 21139 sys_catalog.cc:458] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:39.438658 21148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:39.439268 21148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:52:39 dist-test-slave-tm1g krb5kdc[21040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:39.440927 21148 catalog_manager.cc:1347] Generated new cluster ID: 2160e48db4924bd39a2058a771e8c24f
I20260812 03:52:39.440986 21148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:39.443996 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:39.432637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.17.54.254:35391)
0812 03:52:39.432876 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:39.432885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:39.432923 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:52:39.433432 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:39.433439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:39.433525 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:39.433681 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:39.434397 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.435296 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.436298 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.436811 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.436917 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:39.440202 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:39.440206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:39.440207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:39.440220 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:39.442325 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:39.442739 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:39.442742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:39.442743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:39.442855 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:39.443071 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:39.443072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:39.443072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:39.443362 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:39.443510 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:39.443648 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 03:52:39.443781 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":83,"threads_started":1}
I20260812 03:52:39.456952 21148 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:39.457525 21148 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:39.459856 21148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a05fa836b510440d99c85af1e7317345: Generated new TSK 0
I20260812 03:52:39.460075 21148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:39.494233 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:35391
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:39.597216 21162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:39.597420 21162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:39.597445 21162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:39.598631 21162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:39.598675 21162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:39.598701 21162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:39.600157 21162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:39.600399 21162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:39.600651 21162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:52:39.606328 21162 server_base.cc:1034] running on GCE node
W20260812 03:52:39.606191 21168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:39.606379 21171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:39.606194 21169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:39.606854 21162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:39.607089 21162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:39.608260 21162 hybrid_clock.cc:648] HybridClock initialized: now 1786506759608249 us; error 31 us; skew 500 ppm
Aug 12 03:52:39 dist-test-slave-tm1g krb5kdc[21040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:39.610975 21162 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:39.611459 21162 webserver.cc:460] Webserver started at http://127.17.54.193:42153/ using document root <none> and password file <none>
I20260812 03:52:39.611712 21162 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:39.611784 21162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:39.611945 21162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:39.613159 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "73cbf3f291b844f8be6bdeb6bc649508"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "45d6dc1887f594bcb3654cf0862d38da"
server_key_iv: "734d565c03fa5c2990b9cb64f5479b4a"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.613593 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "73cbf3f291b844f8be6bdeb6bc649508"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "45d6dc1887f594bcb3654cf0862d38da"
server_key_iv: "734d565c03fa5c2990b9cb64f5479b4a"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.615084 21162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:39.616000 21179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.616247 21162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:39.616386 21162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "73cbf3f291b844f8be6bdeb6bc649508"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "45d6dc1887f594bcb3654cf0862d38da"
server_key_iv: "734d565c03fa5c2990b9cb64f5479b4a"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.616468 21162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:39.628404 21162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:39.630864 21162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:39.631062 21162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:39.631348 21162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:39.631699 21162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:39.631760 21162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.631808 21162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:39.631836 21162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.638876 21162 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:33253
I20260812 03:52:39.638926 21306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:33253 every 8 connection(s)
I20260812 03:52:39.639294 21162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:39.641325 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 21162
I20260812 03:52:39.641461 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:39.641687 17627 external_mini_cluster.cc:1428] Setting key 6ffcf632addfbe96994f66daac0712f0
Aug 12 03:52:39 dist-test-slave-tm1g krb5kdc[21040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:39.650694 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:39.640167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:51747 (local address 127.17.54.254:35391)
0812 03:52:39.640285 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:39.640287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:39.640416 (+ 129us) server_negotiation.cc:407] Connection header received
0812 03:52:39.640964 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:39.640966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:39.640977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:39.641041 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:39.642453 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.643010 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.644583 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.644938 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.645028 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:39.647231 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:39.647233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:39.647233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:39.647242 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:39.648965 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:39.649404 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:39.649406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:39.649406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:39.649481 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:39.649828 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:39.649829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:39.649830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:39.649994 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:39.650071 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:39.650466 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 03:52:39.650567 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:52:39.650774 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:39.640112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:35391 (local address 127.17.54.193:51747)
0812 03:52:39.640281 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:39.640325 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:39.640847 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:39.641123 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:39.641129 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:39.642009 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:39.642268 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:39.642275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.643123 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.643124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:39.644474 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:39.644477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.645037 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.645038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:39.645139 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:39.645726 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:39.645736 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:39.647135 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:39.649072 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:39.649080 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:39.649081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:39.649316 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:39.649611 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:39.649612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:39.649612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:39.649746 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:39.650072 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:39.650078 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:39.650212 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 03:52:39.650433 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 03:52:39.650588 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
I20260812 03:52:39.651645 21307 heartbeater.cc:347] Connected to a master server at 127.17.54.254:35391
I20260812 03:52:39.651782 21307 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:39.652011 21307 heartbeater.cc:511] Master 127.17.54.254:35391 requested a full tablet report, sending...
I20260812 03:52:39.652602 21091 ts_manager.cc:194] Registered new tserver with Master: 73cbf3f291b844f8be6bdeb6bc649508 (127.17.54.193:33253)
I20260812 03:52:39.653350 21091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:51747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:39.674754 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:35391
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:39.763468 21312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:39.763700 21312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:39.763724 21312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:39.764919 21312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:39.764967 21312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:39.764995 21312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:39.766619 21312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:39.766884 21312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:39.767138 21312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:39.772429 21321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:39.772459 21319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:39.772429 21318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:39.772724 21312 server_base.cc:1034] running on GCE node
I20260812 03:52:39.772943 21312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:39.773226 21312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:39.774384 21312 hybrid_clock.cc:648] HybridClock initialized: now 1786506759774405 us; error 69 us; skew 500 ppm
Aug 12 03:52:39 dist-test-slave-tm1g krb5kdc[21040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:39.776924 21312 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:39.777437 21312 webserver.cc:460] Webserver started at http://127.17.54.194:42797/ using document root <none> and password file <none>
I20260812 03:52:39.777699 21312 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:39.777776 21312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:39.777938 21312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:39.779084 21312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "21168860c97442b3bf52ebef464d822e"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "9d7ab709ec75324fed77cf6b6f18a971"
server_key_iv: "1331b4dd980d14f822381c035eea1532"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.779451 21312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "21168860c97442b3bf52ebef464d822e"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "9d7ab709ec75324fed77cf6b6f18a971"
server_key_iv: "1331b4dd980d14f822381c035eea1532"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.780896 21312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:52:39.781719 21328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.782002 21312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:39.782115 21312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "21168860c97442b3bf52ebef464d822e"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "9d7ab709ec75324fed77cf6b6f18a971"
server_key_iv: "1331b4dd980d14f822381c035eea1532"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.782195 21312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:39.802734 21312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:39.804975 21312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:39.805166 21312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:39.805442 21312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:39.805792 21312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:39.805847 21312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.805891 21312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:39.805922 21312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.812100 21312 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:36117
I20260812 03:52:39.812178 21454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:36117 every 8 connection(s)
I20260812 03:52:39.812538 21312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:52:39 dist-test-slave-tm1g krb5kdc[21040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:39.821141 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 21312
I20260812 03:52:39.821265 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:39.821516 17627 external_mini_cluster.cc:1428] Setting key b7509d23c65f1865c75de5414532835b
I20260812 03:52:39.823484 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:39.813430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:50653 (local address 127.17.54.254:35391)
0812 03:52:39.813513 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:39.813515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:39.813691 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:52:39.814251 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:39.814252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:39.814263 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:39.814504 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:39.815607 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.816150 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.817295 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.817512 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.817580 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:39.820061 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:39.820062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:39.820062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:39.820071 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:39.821839 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:39.822380 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:39.822381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:39.822382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:39.822461 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:39.822725 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:39.822726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:39.822727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:39.822912 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:39.822988 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:39.823244 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 03:52:39.823349 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:52:39.823501 21458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:39.813354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:35391 (local address 127.17.54.194:50653)
0812 03:52:39.813556 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:39.813600 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:39.814123 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:39.814593 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:39.814598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:39.815273 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:39.815492 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:39.815497 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.816250 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.816251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:39.817191 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:39.817194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.817622 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.817623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:39.817751 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:39.818392 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:39.818400 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:39.819943 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:39.821953 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:39.821961 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:39.821964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:39.822275 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:39.822539 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:39.822540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:39.822540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:39.822650 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:39.822982 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:39.822986 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:39.823108 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 03:52:39.823194 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:52:39.823327 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260812 03:52:39.824106 21455 heartbeater.cc:347] Connected to a master server at 127.17.54.254:35391
I20260812 03:52:39.824225 21455 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:39.824450 21455 heartbeater.cc:511] Master 127.17.54.254:35391 requested a full tablet report, sending...
I20260812 03:52:39.824910 21091 ts_manager.cc:194] Registered new tserver with Master: 21168860c97442b3bf52ebef464d822e (127.17.54.194:36117)
I20260812 03:52:39.825313 21091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:50653
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:39.851758 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:35391
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:39.937693 21461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:39.937932 21461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:39.937958 21461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:39.939340 21461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:39.939386 21461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:39.939424 21461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:39.940935 21461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:35391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:39.941151 21461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:39.941346 21461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:39.946766 21469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:39.946754 21467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:39.946923 21461 server_base.cc:1034] running on GCE node
W20260812 03:52:39.946759 21466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:39.947356 21461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:39.947641 21461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:39.948844 21461 hybrid_clock.cc:648] HybridClock initialized: now 1786506759948820 us; error 42 us; skew 500 ppm
Aug 12 03:52:39 dist-test-slave-tm1g krb5kdc[21040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:39.951761 21461 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:39.952345 21461 webserver.cc:460] Webserver started at http://127.17.54.195:35429/ using document root <none> and password file <none>
I20260812 03:52:39.952597 21461 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:39.952669 21461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:39.952833 21461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:39.953969 21461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "a2fec8f407e44d4e98d3c711d308a1c1"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "d93c76ff73d4731a3b67a4956c1cecaa"
server_key_iv: "316f377b260f5e0d86ee1ff33c90a6b6"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.954444 21461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "a2fec8f407e44d4e98d3c711d308a1c1"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "d93c76ff73d4731a3b67a4956c1cecaa"
server_key_iv: "316f377b260f5e0d86ee1ff33c90a6b6"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.955979 21461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:39.956943 21476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.957183 21461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:39.957293 21461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "a2fec8f407e44d4e98d3c711d308a1c1"
format_stamp: "Formatted at 2026-08-12 03:52:39 on dist-test-slave-tm1g"
server_key: "d93c76ff73d4731a3b67a4956c1cecaa"
server_key_iv: "316f377b260f5e0d86ee1ff33c90a6b6"
server_key_version: "encryptionkey@0"
I20260812 03:52:39.957424 21461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:39.985095 21461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:39.987727 21461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:39.987939 21461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:39.988243 21461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:39.988662 21461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:39.988729 21461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.988768 21461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:39.988817 21461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:39.995309 21461 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:46327
I20260812 03:52:39.995368 21605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:46327 every 8 connection(s)
I20260812 03:52:39.995731 21461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:39.997931 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 21461
I20260812 03:52:39.998013 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:39.998215 17627 external_mini_cluster.cc:1428] Setting key f3165cd559fe5930114d8ebf4636c680
Aug 12 03:52:40 dist-test-slave-tm1g krb5kdc[21040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:40.006503 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:39.996563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:48373 (local address 127.17.54.254:35391)
0812 03:52:39.996635 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:39.996637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:39.996838 (+ 201us) server_negotiation.cc:407] Connection header received
0812 03:52:39.997328 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:39.997329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:39.997339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:39.997412 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:39.998595 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.999417 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:40.000434 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:40.000645 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:40.000709 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:40.003117 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:40.003118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:40.003119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:40.003126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:40.004614 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:40.005172 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:40.005173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:40.005174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:40.005237 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:40.005592 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:40.005593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:40.005593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:40.005742 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:40.005823 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:40.006229 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 03:52:40.006431 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:52:40.006752 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:39.996537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:35391 (local address 127.17.54.195:48373)
0812 03:52:39.996705 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:39.996747 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:39.997238 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:39.997471 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:39.997474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:39.998160 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:39.998470 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:39.998477 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:39.999509 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:39.999510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:40.000332 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:40.000334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:40.000727 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:40.000728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:40.000831 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:40.001468 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:40.001476 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:40.003012 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:40.004698 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:40.004706 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:40.004707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:40.004949 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:40.005368 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:40.005369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:40.005369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:40.005507 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:40.005909 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:40.005923 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:40.006073 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 03:52:40.006172 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:52:40.006403 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260812 03:52:40.007367 21606 heartbeater.cc:347] Connected to a master server at 127.17.54.254:35391
I20260812 03:52:40.007476 21606 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:40.007644 21606 heartbeater.cc:511] Master 127.17.54.254:35391 requested a full tablet report, sending...
I20260812 03:52:40.008085 21091 ts_manager.cc:194] Registered new tserver with Master: a2fec8f407e44d4e98d3c711d308a1c1 (127.17.54.195:46327)
I20260812 03:52:40.008472 21091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:48373
I20260812 03:52:40.010219 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:52:40 dist-test-slave-tm1g krb5kdc[21040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506760, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 03:52:40 dist-test-slave-tm1g krb5kdc[21040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506760, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:40.037674 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:40.029433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.17.54.254:35391)
0812 03:52:40.029508 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:40.029511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:40.029522 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:40.029577 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:40.029578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:40.029589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:40.029637 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:40.030275 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:40.030887 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:40.031697 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:40.031937 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:40.032027 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:40.034902 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:40.034904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:40.034904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:40.034913 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:40.036279 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:40.036663 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:40.036664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:40.036664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:40.036757 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:40.037033 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:40.037034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:40.037035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:40.037229 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:40.037410 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:40.037522 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 03:52:40.037599 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 03:52:40.037964 21091 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32924
I20260812 03:52:40.038550 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 21162
I20260812 03:52:40.051076 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 21312
I20260812 03:52:40.060015 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 21461
I20260812 03:52:40.068888 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 21054
2026-08-12T03:52:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2021 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:40 dist-test-slave-tm1g krb5kdc[21619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:40 dist-test-slave-tm1g krb5kdc[21619](info): set up 2 sockets
Aug 12 03:52:40 dist-test-slave-tm1g krb5kdc[21619](info): commencing operation
krb5kdc: starting...
W20260812 03:52:41.202844 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:41 dist-test-slave-tm1g krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:41.342686 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:45001
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:45001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:41.432375 21634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:41.432670 21634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:41.432698 21634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:41.434203 21634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:41.434280 21634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:41.434319 21634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:41.434334 21634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:41.434347 21634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:41.436213 21634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:45001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:45001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:41.436540 21634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:41.436825 21634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:41.442474 21643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:41.442492 21641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:41.442474 21640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:41.443872 21634 server_base.cc:1034] running on GCE node
I20260812 03:52:41.444267 21634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:41.444662 21634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:41.445881 21634 hybrid_clock.cc:648] HybridClock initialized: now 1786506761445830 us; error 54 us; skew 500 ppm
Aug 12 03:52:41 dist-test-slave-tm1g krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:41.452621 21634 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:41.453249 21634 webserver.cc:460] Webserver started at http://127.17.54.254:39735/ using document root <none> and password file <none>
I20260812 03:52:41.453534 21634 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:41.453610 21634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:41.453773 21634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:41.454978 21634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "8016b588f4ff4c79b36894caed4be83d"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "37322281c01891bce32fb851a3dfb7cb"
server_key_iv: "7a81aa78d2a328e9e8e781082d01d080"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.455353 21634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "8016b588f4ff4c79b36894caed4be83d"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "37322281c01891bce32fb851a3dfb7cb"
server_key_iv: "7a81aa78d2a328e9e8e781082d01d080"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.456856 21634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:52:41.457762 21650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:41.457999 21634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:41.458117 21634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "8016b588f4ff4c79b36894caed4be83d"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "37322281c01891bce32fb851a3dfb7cb"
server_key_iv: "7a81aa78d2a328e9e8e781082d01d080"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.458199 21634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:41.468725 21634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:41.471340 21634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:41.471592 21634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:41.476982 21634 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:45001
I20260812 03:52:41.477023 21706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:45001 every 8 connection(s)
I20260812 03:52:41.477591 21634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:41.478549 21707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:41.479175 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 21634
I20260812 03:52:41.479291 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:41.479475 17627 external_mini_cluster.cc:1428] Setting key 1d1808abea32bb96c905927b89f59de1
I20260812 03:52:41.481966 21707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d: Bootstrap starting.
I20260812 03:52:41.482704 21707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:41.483004 21707 log.cc:824] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:41.483759 21707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d: No bootstrap required, opened a new log
I20260812 03:52:41.485762 21707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8016b588f4ff4c79b36894caed4be83d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45001 } }
I20260812 03:52:41.485944 21707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:41.485997 21707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8016b588f4ff4c79b36894caed4be83d, State: Initialized, Role: FOLLOWER
I20260812 03:52:41.486169 21707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8016b588f4ff4c79b36894caed4be83d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45001 } }
I20260812 03:52:41.486269 21707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:41.486346 21707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:41.486402 21707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 0 FOLLOWER]: Advancing to term 1
Aug 12 03:52:41 dist-test-slave-tm1g krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:41.487116 21707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8016b588f4ff4c79b36894caed4be83d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45001 } }
I20260812 03:52:41.487254 21707 leader_election.cc:302] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8016b588f4ff4c79b36894caed4be83d; no voters:
I20260812 03:52:41.487448 21707 leader_election.cc:288] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:41.487504 21714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:41.487699 21714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [term 1 LEADER]: Becoming Leader. State: Replica: 8016b588f4ff4c79b36894caed4be83d, State: Running, Role: LEADER
I20260812 03:52:41.487754 21707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:41.487828 21714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8016b588f4ff4c79b36894caed4be83d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45001 } }
I20260812 03:52:41.488225 21714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8016b588f4ff4c79b36894caed4be83d. Latest consensus state: current_term: 1 leader_uuid: "8016b588f4ff4c79b36894caed4be83d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8016b588f4ff4c79b36894caed4be83d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45001 } } }
I20260812 03:52:41.488314 21714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:41.488519 21716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8016b588f4ff4c79b36894caed4be83d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8016b588f4ff4c79b36894caed4be83d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45001 } } }
I20260812 03:52:41.488575 21723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:41.488678 21716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:41.489277 21723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:41.491679 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:41.480123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39216 (local address 127.17.54.254:45001)
0812 03:52:41.480299 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:41.480311 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:41.480349 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:52:41.480776 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:41.480782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:41.480837 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:41.480955 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:41.481876 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.483041 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.483825 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.484252 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.484389 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:41.487088 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:41.487096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:41.487097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:41.487117 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:41.489599 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:41.490170 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:41.490174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:41.490174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:41.490287 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:41.490584 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:41.490585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:41.490586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:41.490930 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:41.491093 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:41.491214 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 03:52:41.491367 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260812 03:52:41.492437 21723 catalog_manager.cc:1347] Generated new cluster ID: bca67beda55c48aabef51fbab902b99b
I20260812 03:52:41.492542 21723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:41.505434 21723 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:41.506475 21723 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:41.513801 21723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8016b588f4ff4c79b36894caed4be83d: Generated new TSK 0
I20260812 03:52:41.514087 21723 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:41.551029 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:45001
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:41.637960 21738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:41.638386 21738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:41.638447 21738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:41.639648 21738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:41.639690 21738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:41.639716 21738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:41.641609 21738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:41.641839 21738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:41.642037 21738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:41.647254 21748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:41.647239 21743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:41.647442 21745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:41.647686 21738 server_base.cc:1034] running on GCE node
I20260812 03:52:41.647929 21738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:41.648212 21738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:41.649361 21738 hybrid_clock.cc:648] HybridClock initialized: now 1786506761649350 us; error 33 us; skew 500 ppm
Aug 12 03:52:41 dist-test-slave-tm1g krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:41.652053 21738 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:41.652542 21738 webserver.cc:460] Webserver started at http://127.17.54.193:34415/ using document root <none> and password file <none>
I20260812 03:52:41.652806 21738 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:41.652874 21738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:41.653026 21738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:41.654101 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "f527835ecf174e5bb5555bff3f7d494f"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "9e3788be77a1621abedb2cdec5c20af7"
server_key_iv: "e2b1c03379fec1d63ce09aefc3e20af2"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.654491 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "f527835ecf174e5bb5555bff3f7d494f"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "9e3788be77a1621abedb2cdec5c20af7"
server_key_iv: "e2b1c03379fec1d63ce09aefc3e20af2"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.655963 21738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:41.656922 21758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:41.657130 21738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:41.657219 21738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "f527835ecf174e5bb5555bff3f7d494f"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "9e3788be77a1621abedb2cdec5c20af7"
server_key_iv: "e2b1c03379fec1d63ce09aefc3e20af2"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.657279 21738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:41.668231 21738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:41.670758 21738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:41.670887 21738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:41.671092 21738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:41.671450 21738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:41.671489 21738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:41.671514 21738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:41.671550 21738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:41.678261 21738 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:46563
I20260812 03:52:41.678362 21890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:46563 every 8 connection(s)
I20260812 03:52:41.678715 21738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:52:41 dist-test-slave-tm1g krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:41.687634 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 21738
I20260812 03:52:41.687780 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:41.688041 17627 external_mini_cluster.cc:1428] Setting key b41da2945d8b483094f106f4efe820dd
I20260812 03:52:41.689875 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:41.679702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:46975 (local address 127.17.54.254:45001)
0812 03:52:41.679784 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:41.679786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:41.680015 (+ 229us) server_negotiation.cc:407] Connection header received
0812 03:52:41.681038 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:41.681040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:41.681055 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:41.681102 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:41.681858 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.682466 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.683546 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.683779 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.683868 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:41.686270 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:41.686272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:41.686273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:41.686287 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:41.688014 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:41.688644 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:41.688645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:41.688645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:41.688736 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:41.689063 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:41.689064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:41.689064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:41.689193 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:41.689251 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:41.689688 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 03:52:41.689807 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:52:41.690093 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:41.679672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:45001 (local address 127.17.54.193:46975)
0812 03:52:41.679867 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:41.679908 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:41.680894 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:41.681192 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:41.681196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:41.681525 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:41.681734 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:41.681739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.682575 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.682576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:41.683448 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:41.683451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.683886 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.683887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:41.684033 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:41.684614 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:41.684624 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:41.686160 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:41.688110 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:41.688120 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:41.688123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:41.688478 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:41.688842 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:41.688843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:41.688844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:41.688996 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:41.689266 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:41.689273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:41.689454 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 03:52:41.689590 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 03:52:41.689799 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":74,"threads_started":1}
I20260812 03:52:41.690636 21891 heartbeater.cc:347] Connected to a master server at 127.17.54.254:45001
I20260812 03:52:41.690760 21891 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:41.690997 21891 heartbeater.cc:511] Master 127.17.54.254:45001 requested a full tablet report, sending...
I20260812 03:52:41.691636 21668 ts_manager.cc:194] Registered new tserver with Master: f527835ecf174e5bb5555bff3f7d494f (127.17.54.193:46563)
I20260812 03:52:41.692370 21668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:46975
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:41.718520 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:45001
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:41.809911 21897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:41.810220 21897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:41.810268 21897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:41.811591 21897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:41.811671 21897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:41.811718 21897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:41.813616 21897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:41.813972 21897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:41.814251 21897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:41.820813 21906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:41.820813 21909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:41.821184 21907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:41.821385 21897 server_base.cc:1034] running on GCE node
I20260812 03:52:41.821592 21897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:41.821821 21897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:41.823019 21897 hybrid_clock.cc:648] HybridClock initialized: now 1786506761823002 us; error 32 us; skew 500 ppm
Aug 12 03:52:41 dist-test-slave-tm1g krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:41.825953 21897 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:41.826467 21897 webserver.cc:460] Webserver started at http://127.17.54.194:33149/ using document root <none> and password file <none>
I20260812 03:52:41.826699 21897 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:41.826747 21897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:41.826848 21897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:41.827946 21897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "dd0109a16a6148d99f98bcbb3cf9ad35"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "063560745eede4da165a3dca2d8a8431"
server_key_iv: "e464ba45d4660fc9486e4f062326bff6"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.828387 21897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "dd0109a16a6148d99f98bcbb3cf9ad35"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "063560745eede4da165a3dca2d8a8431"
server_key_iv: "e464ba45d4660fc9486e4f062326bff6"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.829944 21897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:41.831254 21917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:41.831563 21897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:41.831666 21897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "dd0109a16a6148d99f98bcbb3cf9ad35"
format_stamp: "Formatted at 2026-08-12 03:52:41 on dist-test-slave-tm1g"
server_key: "063560745eede4da165a3dca2d8a8431"
server_key_iv: "e464ba45d4660fc9486e4f062326bff6"
server_key_version: "encryptionkey@0"
I20260812 03:52:41.831729 21897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:41.861621 21897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:41.863961 21897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:41.864089 21897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:41.864293 21897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:41.864637 21897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:41.864675 21897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:41.864698 21897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:41.864711 21897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:41.870560 21897 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:38293
I20260812 03:52:41.870659 22055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:38293 every 8 connection(s)
I20260812 03:52:41.870977 21897 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:52:41.875370 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 21897
I20260812 03:52:41.875495 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:41.875710 17627 external_mini_cluster.cc:1428] Setting key 2c1f4a5e74c7cef03c7017e007a0ae1b
Aug 12 03:52:41 dist-test-slave-tm1g krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:41.882123 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:41.871797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:50265 (local address 127.17.54.254:45001)
0812 03:52:41.871906 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:41.871909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:41.872071 (+ 162us) server_negotiation.cc:407] Connection header received
0812 03:52:41.873091 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:41.873092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:41.873106 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:41.873164 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:41.873832 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.874429 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.875672 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.875984 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.876056 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:41.878795 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:41.878797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:41.878797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:41.878805 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:41.880577 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:41.881031 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:41.881033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:41.881034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:41.881107 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:41.881367 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:41.881368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:41.881369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:41.881533 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:41.881617 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:41.881918 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 03:52:41.882019 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:52:41.882387 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:41.871735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:45001 (local address 127.17.54.194:50265)
0812 03:52:41.871946 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:41.871984 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:41.872973 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:41.873239 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:41.873243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:41.873523 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:41.873729 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:41.873733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.874524 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.874525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:41.875577 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:41.875581 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:41.876076 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:41.876077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:41.876224 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:41.876857 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:41.876869 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:41.878694 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:41.880678 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:41.880685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:41.880688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:41.880950 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:41.881179 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:41.881180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:41.881180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:41.881285 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:41.881608 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:41.881617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:41.881773 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 03:52:41.881881 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:52:41.882088 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260812 03:52:41.882923 22056 heartbeater.cc:347] Connected to a master server at 127.17.54.254:45001
I20260812 03:52:41.883052 22056 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:41.883294 22056 heartbeater.cc:511] Master 127.17.54.254:45001 requested a full tablet report, sending...
I20260812 03:52:41.883759 21668 ts_manager.cc:194] Registered new tserver with Master: dd0109a16a6148d99f98bcbb3cf9ad35 (127.17.54.194:38293)
I20260812 03:52:41.884161 21668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:50265
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:41.906572 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:45001
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:41.993912 22063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:41.994134 22063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:41.994160 22063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:41.995405 22063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:41.995457 22063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:41.995486 22063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:41.997036 22063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:41.997252 22063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:41.997464 22063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:42.003088 22072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:42.003113 22074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:42.003122 22071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:42.003263 22063 server_base.cc:1034] running on GCE node
I20260812 03:52:42.003662 22063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:42.003911 22063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:42.005079 22063 hybrid_clock.cc:648] HybridClock initialized: now 1786506762005057 us; error 44 us; skew 500 ppm
Aug 12 03:52:42 dist-test-slave-tm1g krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506762, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:42.008159 22063 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:42.008766 22063 webserver.cc:460] Webserver started at http://127.17.54.195:33333/ using document root <none> and password file <none>
I20260812 03:52:42.009064 22063 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:42.009136 22063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:42.009289 22063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:42.010526 22063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "34d1e2ebe9934b28ad71a4a0a133f2e4"
format_stamp: "Formatted at 2026-08-12 03:52:42 on dist-test-slave-tm1g"
server_key: "c5548eca018ec181cd272b5f322a1eea"
server_key_iv: "cfc594427913f4e3fb4d8799d9b0a545"
server_key_version: "encryptionkey@0"
I20260812 03:52:42.010914 22063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "34d1e2ebe9934b28ad71a4a0a133f2e4"
format_stamp: "Formatted at 2026-08-12 03:52:42 on dist-test-slave-tm1g"
server_key: "c5548eca018ec181cd272b5f322a1eea"
server_key_iv: "cfc594427913f4e3fb4d8799d9b0a545"
server_key_version: "encryptionkey@0"
I20260812 03:52:42.012543 22063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:42.013446 22085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:42.013643 22063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:42.013752 22063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "34d1e2ebe9934b28ad71a4a0a133f2e4"
format_stamp: "Formatted at 2026-08-12 03:52:42 on dist-test-slave-tm1g"
server_key: "c5548eca018ec181cd272b5f322a1eea"
server_key_iv: "cfc594427913f4e3fb4d8799d9b0a545"
server_key_version: "encryptionkey@0"
I20260812 03:52:42.013868 22063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:42.031853 22063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:42.034214 22063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:42.034480 22063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:42.034775 22063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:42.035138 22063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:42.035176 22063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:42.035202 22063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:42.035213 22063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:42.042177 22063 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:33069
I20260812 03:52:42.042222 22209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:33069 every 8 connection(s)
I20260812 03:52:42.042730 22063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:42.042887 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 22063
I20260812 03:52:42.043006 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:42.043226 17627 external_mini_cluster.cc:1428] Setting key ef7ea4e02ba4ebabe70d0175180034c0
Aug 12 03:52:42 dist-test-slave-tm1g krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506762, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:42.055276 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:42.043598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:37689 (local address 127.17.54.254:45001)
0812 03:52:42.043654 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:42.043657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:42.043975 (+ 318us) server_negotiation.cc:407] Connection header received
0812 03:52:42.045039 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:42.045040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:42.045051 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:42.045117 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:42.045821 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:42.046676 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:42.047832 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:42.048038 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:42.048105 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:42.051075 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:42.051077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:42.051077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:42.051087 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:42.053031 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:42.053618 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:42.053619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:42.053619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:42.053751 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:42.054279 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:42.054280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:42.054281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:42.054542 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:42.054712 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:42.055044 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 03:52:42.055163 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:52:42.055512 22214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:42.043617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:45001 (local address 127.17.54.195:37689)
0812 03:52:42.043856 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:42.043907 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:42.044917 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:42.045182 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:42.045186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:42.045483 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:42.045698 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:42.045703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:42.046814 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:42.046815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:42.047701 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:42.047704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:42.048183 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:42.048186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:42.048423 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:42.049133 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:42.049143 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:42.050934 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:42.053185 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:42.053195 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:42.053198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:42.053545 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:42.053874 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:42.053875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:42.053875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:42.054156 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:42.054642 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:42.054651 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:42.054894 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 03:52:42.054995 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:52:42.055233 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":94,"threads_started":1}
I20260812 03:52:42.056010 22210 heartbeater.cc:347] Connected to a master server at 127.17.54.254:45001
I20260812 03:52:42.056131 22210 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:42.056327 22210 heartbeater.cc:511] Master 127.17.54.254:45001 requested a full tablet report, sending...
I20260812 03:52:42.056713 21668 ts_manager.cc:194] Registered new tserver with Master: 34d1e2ebe9934b28ad71a4a0a133f2e4 (127.17.54.195:33069)
I20260812 03:52:42.057111 21668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:37689
I20260812 03:52:42.066715 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:52:42 dist-test-slave-tm1g krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:52:42 dist-test-slave-tm1g krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:42.093302 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:42.084336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.17.54.193:46563)
0812 03:52:42.084610 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:42.084615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:42.084627 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:42.084657 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:42.084659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:42.084675 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:42.084760 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:42.085396 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:42.086167 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:42.087034 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:42.087368 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:42.087438 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:42.090358 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:42.090373 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:42.090373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:42.090381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:42.092093 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:42.092447 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:42.092449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:42.092450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:42.092520 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:42.092734 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:42.092734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:42.092735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:42.092878 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:42.092986 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:42.093116 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 03:52:42.093181 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":114,"threads_started":1}
Aug 12 03:52:42 dist-test-slave-tm1g krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:42.102586 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:42.094976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.17.54.254:45001)
0812 03:52:42.095068 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:42.095070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:42.095082 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:42.095155 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:42.095156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:42.095166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:42.095215 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:42.095776 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:42.096330 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:42.097041 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:42.097239 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:42.097312 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:42.099850 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:42.099852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:42.099852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:42.099860 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:42.101381 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:42.101736 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:42.101737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:42.101737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:42.101791 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:42.102013 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:42.102013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:42.102014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:42.102166 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:42.102246 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:42.102446 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 03:52:42.102513 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 03:52:42.102929 21668 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39226
I20260812 03:52:42.103554 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 21738
I20260812 03:52:42.112223 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 21897
I20260812 03:52:42.122942 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 22063
I20260812 03:52:42.130532 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 21634
2026-08-12T03:52:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2060 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:42 dist-test-slave-tm1g krb5kdc[22226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:42 dist-test-slave-tm1g krb5kdc[22226](info): set up 2 sockets
Aug 12 03:52:42 dist-test-slave-tm1g krb5kdc[22226](info): commencing operation
krb5kdc: starting...
W20260812 03:52:43.255266 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:43 dist-test-slave-tm1g krb5kdc[22226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:43.398612 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34799
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:34799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:43.491415 22242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:43.491674 22242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:43.491703 22242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:43.492954 22242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:43.493002 22242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:43.493017 22242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:43.493036 22242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:43.493049 22242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:43.494706 22242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:34799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:43.494939 22242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:43.495131 22242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:43.500754 22248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:43.500767 22250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:43.500767 22247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:43.500809 22242 server_base.cc:1034] running on GCE node
I20260812 03:52:43.501259 22242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:43.501576 22242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:43.502754 22242 hybrid_clock.cc:648] HybridClock initialized: now 1786506763502742 us; error 46 us; skew 500 ppm
Aug 12 03:52:43 dist-test-slave-tm1g krb5kdc[22226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506763, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:43.505388 22242 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:43.505757 22242 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:43.505807 22242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:43.505901 22242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:43.507014 22242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "c13ca25eb0234ed38bf5efdd5fd10112"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "8bd1aa11e2812b7354cd9d675fb48ef5"
server_key_iv: "180d49b94529d07395522011de8cf4c7"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.507340 22242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "c13ca25eb0234ed38bf5efdd5fd10112"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "8bd1aa11e2812b7354cd9d675fb48ef5"
server_key_iv: "180d49b94529d07395522011de8cf4c7"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.508632 22242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:52:43.509321 22258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:43.509511 22242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:43.509605 22242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "c13ca25eb0234ed38bf5efdd5fd10112"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "8bd1aa11e2812b7354cd9d675fb48ef5"
server_key_iv: "180d49b94529d07395522011de8cf4c7"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.509658 22242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:43.535274 22242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:43.537761 22242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:43.537887 22242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:43.542109 22242 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:34799
I20260812 03:52:43.542140 22318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:34799 every 8 connection(s)
I20260812 03:52:43.542467 22242 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:43.543681 22320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:43.545213 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 22242
I20260812 03:52:43.545310 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:43.545513 17627 external_mini_cluster.cc:1428] Setting key a1fb803bc8ab01597ee7b74d759ea4df
I20260812 03:52:43.546492 22320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112: Bootstrap starting.
I20260812 03:52:43.547207 22320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:43.547573 22320 log.cc:824] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:43.548360 22320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112: No bootstrap required, opened a new log
I20260812 03:52:43.550621 22320 raft_consensus.cc:348] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ca25eb0234ed38bf5efdd5fd10112" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34799 } }
I20260812 03:52:43.550810 22320 raft_consensus.cc:374] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:43.550855 22320 raft_consensus.cc:729] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13ca25eb0234ed38bf5efdd5fd10112, State: Initialized, Role: FOLLOWER
I20260812 03:52:43.551033 22320 consensus_queue.cc:260] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ca25eb0234ed38bf5efdd5fd10112" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34799 } }
I20260812 03:52:43.551122 22320 raft_consensus.cc:388] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:43.551167 22320 raft_consensus.cc:482] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:43.551215 22320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:43.551793 22320 raft_consensus.cc:504] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ca25eb0234ed38bf5efdd5fd10112" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34799 } }
I20260812 03:52:43.551918 22320 leader_election.cc:302] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c13ca25eb0234ed38bf5efdd5fd10112; no voters:
I20260812 03:52:43.552111 22320 leader_election.cc:288] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:43.552143 22325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:43.552268 22325 raft_consensus.cc:686] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [term 1 LEADER]: Becoming Leader. State: Replica: c13ca25eb0234ed38bf5efdd5fd10112, State: Running, Role: LEADER
I20260812 03:52:43.552394 22320 sys_catalog.cc:564] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:43.552421 22325 consensus_queue.cc:237] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ca25eb0234ed38bf5efdd5fd10112" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34799 } }
Aug 12 03:52:43 dist-test-slave-tm1g krb5kdc[22226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:43.552861 22326 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c13ca25eb0234ed38bf5efdd5fd10112. Latest consensus state: current_term: 1 leader_uuid: "c13ca25eb0234ed38bf5efdd5fd10112" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ca25eb0234ed38bf5efdd5fd10112" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34799 } } }
I20260812 03:52:43.552927 22327 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c13ca25eb0234ed38bf5efdd5fd10112" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ca25eb0234ed38bf5efdd5fd10112" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34799 } } }
I20260812 03:52:43.553071 22327 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:43.553061 22326 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:43.553352 22340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:43.554257 22340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:43.556533 22340 catalog_manager.cc:1347] Generated new cluster ID: dd63c63ee5cb4d8580dafed0191db85b
I20260812 03:52:43.556586 22340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:43.557525 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:43.546070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33126 (local address 127.17.54.254:34799)
0812 03:52:43.546244 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:43.546252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:43.546277 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:52:43.546810 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:43.546817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:43.546868 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:43.546991 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:43.547864 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.548954 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.550050 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.550512 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.550593 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:43.553380 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:43.553385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:43.553385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:43.553398 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:43.555532 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:43.555919 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:43.555923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:43.555923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:43.556032 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:43.556299 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:43.556300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:43.556300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:43.556624 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:43.556747 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:43.557184 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 03:52:43.557314 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260812 03:52:43.563665 22340 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:43.564205 22340 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:43.569453 22340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112: Generated new TSK 0
I20260812 03:52:43.569711 22340 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:43.603533 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34799
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:43.690832 22353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:43.691174 22353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:43.691217 22353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:43.692691 22353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:43.692766 22353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:43.692816 22353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:43.694497 22353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:43.694773 22353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:43.695021 22353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:43.700816 22363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:43.700839 22360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:43.700816 22359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:43.701050 22353 server_base.cc:1034] running on GCE node
I20260812 03:52:43.701483 22353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:43.701799 22353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:43.702978 22353 hybrid_clock.cc:648] HybridClock initialized: now 1786506763702966 us; error 33 us; skew 500 ppm
Aug 12 03:52:43 dist-test-slave-tm1g krb5kdc[22226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506763, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:43.705713 22353 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:43.706104 22353 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:43.706190 22353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:43.706395 22353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:43.707562 22353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "a6f2ea883f8746aab90c6d02a31cf1da"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "cd8f78d526e542b1547d78911a15bd6d"
server_key_iv: "4c7aa560f59beb644fa4c3bf49d2b79b"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.707957 22353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "a6f2ea883f8746aab90c6d02a31cf1da"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "cd8f78d526e542b1547d78911a15bd6d"
server_key_iv: "4c7aa560f59beb644fa4c3bf49d2b79b"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.709438 22353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:43.710362 22369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:43.710578 22353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:43.710675 22353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "a6f2ea883f8746aab90c6d02a31cf1da"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "cd8f78d526e542b1547d78911a15bd6d"
server_key_iv: "4c7aa560f59beb644fa4c3bf49d2b79b"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.710737 22353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:43.730612 22353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:43.732998 22353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:43.733120 22353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:43.733338 22353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:43.733618 22353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:43.733649 22353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:43.733673 22353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:43.733686 22353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:43.739648 22353 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:33707
I20260812 03:52:43.739699 22499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:33707 every 8 connection(s)
I20260812 03:52:43.740073 22353 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:43.750375 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 22353
I20260812 03:52:43.750480 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:43.750634 17627 external_mini_cluster.cc:1428] Setting key e7a552ff0ccf689b7e5752bb303f9747
Aug 12 03:52:43 dist-test-slave-tm1g krb5kdc[22226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506763, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:43.755142 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:43.743036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:53479 (local address 127.17.54.254:34799)
0812 03:52:43.743122 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:43.743124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:43.744902 (+ 1778us) server_negotiation.cc:407] Connection header received
0812 03:52:43.745620 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:43.745621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:43.745635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:43.745707 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:43.747112 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.747870 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.749361 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.749612 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.749677 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:43.752091 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:43.752093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:43.752094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:43.752102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:43.753633 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:43.754052 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:43.754053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:43.754053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:43.754118 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:43.754462 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:43.754463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:43.754463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:43.754588 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:43.754664 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:43.754955 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 03:52:43.755047 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:52:43.755352 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:43.743003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34799 (local address 127.17.54.193:53479)
0812 03:52:43.744726 (+ 1723us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:43.744783 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:43.745489 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:43.745880 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:43.745885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:43.746328 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:43.747010 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:43.747016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.747993 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.747994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:43.749248 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:43.749251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.749702 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.749703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:43.749847 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:43.750433 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:43.750457 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:43.751996 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:43.753725 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:43.753733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:43.753735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:43.753976 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:43.754190 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:43.754191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:43.754191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:43.754388 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:43.754655 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:43.754662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:43.754812 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 03:52:43.754921 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:52:43.755088 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1647,"thread_start_us":82,"threads_started":1}
I20260812 03:52:43.755800 22500 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34799
I20260812 03:52:43.755900 22500 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:43.756078 22500 heartbeater.cc:511] Master 127.17.54.254:34799 requested a full tablet report, sending...
I20260812 03:52:43.756585 22279 ts_manager.cc:194] Registered new tserver with Master: a6f2ea883f8746aab90c6d02a31cf1da (127.17.54.193:33707)
I20260812 03:52:43.757198 22279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:53479
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:43.779610 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34799
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:43.870785 22505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:43.871004 22505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:43.871030 22505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:43.872159 22505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:43.872195 22505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:43.872222 22505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:43.873728 22505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:43.873921 22505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:43.874110 22505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:43.879914 22511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:43.879926 22513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:43.879937 22510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:43.880034 22505 server_base.cc:1034] running on GCE node
I20260812 03:52:43.880379 22505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:43.880630 22505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:43.881824 22505 hybrid_clock.cc:648] HybridClock initialized: now 1786506763881814 us; error 48 us; skew 500 ppm
Aug 12 03:52:43 dist-test-slave-tm1g krb5kdc[22226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506763, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:43.884657 22505 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:43.885002 22505 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:43.885048 22505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:43.885145 22505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:43.886255 22505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "edc03ebfe7414468800e443de134ad5f"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "82b599d6eabb94f7f1b475581656bd94"
server_key_iv: "23a1899141ca2332d9d19bad120424d5"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.886674 22505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "edc03ebfe7414468800e443de134ad5f"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "82b599d6eabb94f7f1b475581656bd94"
server_key_iv: "23a1899141ca2332d9d19bad120424d5"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.888073 22505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:43.888931 22521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:43.889106 22505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:43.889189 22505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "edc03ebfe7414468800e443de134ad5f"
format_stamp: "Formatted at 2026-08-12 03:52:43 on dist-test-slave-tm1g"
server_key: "82b599d6eabb94f7f1b475581656bd94"
server_key_iv: "23a1899141ca2332d9d19bad120424d5"
server_key_version: "encryptionkey@0"
I20260812 03:52:43.889245 22505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:43.908579 22505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:43.910862 22505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:43.911026 22505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:43.911254 22505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:43.911612 22505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:43.911649 22505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:43.911674 22505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:43.911687 22505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:43.917903 22505 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:35339
I20260812 03:52:43.917980 22649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:35339 every 8 connection(s)
I20260812 03:52:43.918265 22505 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:52:43 dist-test-slave-tm1g krb5kdc[22226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506763, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:43.926080 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 22505
I20260812 03:52:43.926209 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:43.926481 17627 external_mini_cluster.cc:1428] Setting key a89fb3fcc091bedddb9e5f723c7c97be
I20260812 03:52:43.928625 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:43.919155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:40957 (local address 127.17.54.254:34799)
0812 03:52:43.919232 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:43.919235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:43.919442 (+ 207us) server_negotiation.cc:407] Connection header received
0812 03:52:43.920025 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:43.920027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:43.920040 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:43.920106 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:43.921102 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.921631 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.922687 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.922895 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.922975 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:43.925242 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:43.925244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:43.925244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:43.925253 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:43.927017 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:43.927555 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:43.927556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:43.927557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:43.927627 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:43.927867 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:43.927868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:43.927869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:43.928048 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:43.928130 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:43.928385 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 03:52:43.928482 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:52:43.928655 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:43.919138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34799 (local address 127.17.54.194:40957)
0812 03:52:43.919303 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:43.919348 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:43.919914 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:43.920182 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:43.920186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:43.920465 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:43.920999 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:43.921005 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.921735 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.921736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:43.922596 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:43.922598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:43.922982 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:43.922983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:43.923110 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:43.923668 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:43.923676 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:43.925128 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:43.927113 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:43.927120 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:43.927123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:43.927467 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:43.927699 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:43.927700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:43.927700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:43.927806 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:43.928125 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:43.928130 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:43.928255 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 03:52:43.928344 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:52:43.928472 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260812 03:52:43.929240 22650 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34799
I20260812 03:52:43.929358 22650 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:43.929523 22650 heartbeater.cc:511] Master 127.17.54.254:34799 requested a full tablet report, sending...
I20260812 03:52:43.929927 22279 ts_manager.cc:194] Registered new tserver with Master: edc03ebfe7414468800e443de134ad5f (127.17.54.194:35339)
I20260812 03:52:43.930383 22279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:40957
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:43.957196 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34799
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:44.041450 22655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:44.041683 22655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:44.041710 22655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:44.043030 22655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:44.043081 22655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:44.043112 22655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:44.044879 22655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:44.045166 22655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:44.045451 22655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:44.051283 22660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:44.051565 22661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:44.051692 22663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:44.051749 22655 server_base.cc:1034] running on GCE node
I20260812 03:52:44.052140 22655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:44.052461 22655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:44.053866 22655 hybrid_clock.cc:648] HybridClock initialized: now 1786506764053844 us; error 49 us; skew 500 ppm
Aug 12 03:52:44 dist-test-slave-tm1g krb5kdc[22226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506764, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:44.056977 22655 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:44.057466 22655 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:44.057544 22655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:44.057678 22655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:44.058929 22655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "aff27111f6274a398ad71c3850fa0187"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "15ca0333a334a6b2bbd9c1508019752b"
server_key_iv: "19f0c4ec537505da6c0787f4593b563e"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.059296 22655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "aff27111f6274a398ad71c3850fa0187"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "15ca0333a334a6b2bbd9c1508019752b"
server_key_iv: "19f0c4ec537505da6c0787f4593b563e"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.060765 22655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 03:52:44.061708 22671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.061923 22655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:44.062039 22655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "aff27111f6274a398ad71c3850fa0187"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "15ca0333a334a6b2bbd9c1508019752b"
server_key_iv: "19f0c4ec537505da6c0787f4593b563e"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.062143 22655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:44.074698 22655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:44.077181 22655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:44.077360 22655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:44.077607 22655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:44.077992 22655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:44.078049 22655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.078090 22655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:44.078121 22655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.085069 22655 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:45511
I20260812 03:52:44.085109 22794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:45511 every 8 connection(s)
I20260812 03:52:44.085435 22655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:52:44 dist-test-slave-tm1g krb5kdc[22226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506764, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:44.093384 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 22655
I20260812 03:52:44.093513 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:44.093737 17627 external_mini_cluster.cc:1428] Setting key 3fe02919891e8c9891f3eb7aaa335f01
I20260812 03:52:44.095986 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.086415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:45463 (local address 127.17.54.254:34799)
0812 03:52:44.086488 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.086491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.086614 (+ 123us) server_negotiation.cc:407] Connection header received
0812 03:52:44.087138 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.087139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.087151 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.087215 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.088211 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.088736 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.089789 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.090001 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.090087 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.092553 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.092554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.092555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:44.092563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.094482 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:44.095059 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:44.095060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:44.095060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:44.095118 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:44.095329 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:44.095330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:44.095330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:44.095459 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.095537 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.095836 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.095918 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:52:44.096237 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.086233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34799 (local address 127.17.54.195:45463)
0812 03:52:44.086461 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.086513 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.087027 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.087288 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.087291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.087566 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.088120 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.088126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.088845 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.088847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.089700 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.089702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.090123 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.090124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.090248 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.090866 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:44.090875 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.092444 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.094591 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:44.094604 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:44.094607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:44.094967 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:44.095160 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:44.095160 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:44.095161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:44.095258 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:44.095551 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.095558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.095693 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.095798 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.095973 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":120,"threads_started":1}
I20260812 03:52:44.096714 22795 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34799
I20260812 03:52:44.096807 22795 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:44.097025 22795 heartbeater.cc:511] Master 127.17.54.254:34799 requested a full tablet report, sending...
I20260812 03:52:44.097397 22279 ts_manager.cc:194] Registered new tserver with Master: aff27111f6274a398ad71c3850fa0187 (127.17.54.195:45511)
I20260812 03:52:44.097816 22279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:45463
I20260812 03:52:44.106227 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:52:44.114893 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.107885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.17.54.254:34799)
0812 03:52:44.107946 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.107949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.108093 (+ 144us) server_negotiation.cc:407] Connection header received
0812 03:52:44.108179 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.108180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.108190 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.108259 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.108891 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.109422 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.110217 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.110506 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.110581 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.111835 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.111837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.111837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:44.111845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.113570 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:44.113926 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:44.113927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:44.113928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:44.113985 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:44.114208 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:44.114208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:44.114209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:44.114476 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.114590 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.114701 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.114787 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:52:44.116748 22279 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:44.118546 22279 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:44.125682 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.121522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:45511 (local address 127.0.0.1:58948)
0812 03:52:44.121931 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.121943 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.121991 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.122230 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.122231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.122233 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:44.122288 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.122290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.123525 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.123527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.124555 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.124557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.125443 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.125444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.125508 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.125519 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.125578 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.125619 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":35,"thread_start_us":64,"threads_started":1}
I20260812 03:52:44.125758 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.121743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.17.54.195:45511)
0812 03:52:44.122059 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.122064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.122073 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:44.122114 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.122115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.122132 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.122232 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:44.122386 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.123426 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.124645 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.125334 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.125414 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.125474 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.125609 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.125652 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":84,"threads_started":1}
I20260812 03:52:44.126091 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.121241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33707 (local address 127.0.0.1:57450)
0812 03:52:44.121490 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.121564 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.121635 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.121870 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.121874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.121876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:44.121958 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.121961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.123363 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.123364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.124810 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.124812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.125760 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.125761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.125889 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.125895 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.125982 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.126026 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":103,"thread_start_us":49,"threads_started":1}
I20260812 03:52:44.126435 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.121354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.17.54.193:33707)
0812 03:52:44.121670 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.121675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.121691 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:52:44.121738 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.121741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.121758 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.121858 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:44.122053 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.123249 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.124900 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.125662 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.125733 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.125786 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.126015 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.126057 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":110,"threads_started":1}
I20260812 03:52:44.126922 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.121432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38168 (local address 127.17.54.194:35339)
0812 03:52:44.122400 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.122405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.122458 (+ 53us) server_negotiation.cc:407] Connection header received
0812 03:52:44.122490 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.122492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.122503 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.122589 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:44.122706 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.123454 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.125829 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.126622 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.126673 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.126707 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.126834 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.126864 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":927,"thread_start_us":94,"threads_started":1}
I20260812 03:52:44.126956 22725 tablet_service.cc:1467] Processing CreateTablet for tablet fc0c91405f284067850e04ff133e96e8 (DEFAULT_TABLE table=test-table [id=a5c33da7693246b4ba991c7ba8f0a0d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:44.127343 22725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0c91405f284067850e04ff133e96e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:44.126935 22807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.121295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:35339 (local address 127.0.0.1:38168)
0812 03:52:44.121549 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.121565 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.121635 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.122590 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.122591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.122593 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:44.122635 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.122636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.123571 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.123572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.124809 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.124812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.126698 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.126699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.126763 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.126767 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.126826 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.126866 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":28,"thread_start_us":120,"threads_started":1}
I20260812 03:52:44.130173 22814 tablet_bootstrap.cc:492] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Bootstrap starting.
I20260812 03:52:44.130895 22425 tablet_service.cc:1467] Processing CreateTablet for tablet fc0c91405f284067850e04ff133e96e8 (DEFAULT_TABLE table=test-table [id=a5c33da7693246b4ba991c7ba8f0a0d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:44.130997 22574 tablet_service.cc:1467] Processing CreateTablet for tablet fc0c91405f284067850e04ff133e96e8 (DEFAULT_TABLE table=test-table [id=a5c33da7693246b4ba991c7ba8f0a0d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:44.131285 22574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0c91405f284067850e04ff133e96e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:44.131285 22425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0c91405f284067850e04ff133e96e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:44.134702 22816 tablet_bootstrap.cc:492] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da: Bootstrap starting.
I20260812 03:52:44.135090 22814 tablet_bootstrap.cc:654] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:44.135362 22816 tablet_bootstrap.cc:654] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:44.135679 22814 log.cc:824] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:44.135927 22816 log.cc:824] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:44.136313 22815 tablet_bootstrap.cc:492] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f: Bootstrap starting.
I20260812 03:52:44.136561 22814 tablet_bootstrap.cc:492] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: No bootstrap required, opened a new log
I20260812 03:52:44.136620 22814 ts_tablet_manager.cc:1397] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 03:52:44.137102 22815 tablet_bootstrap.cc:654] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:44.137620 22815 log.cc:824] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:44.138211 22816 tablet_bootstrap.cc:492] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da: No bootstrap required, opened a new log
I20260812 03:52:44.138319 22816 ts_tablet_manager.cc:1397] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:52:44.138355 22814 raft_consensus.cc:348] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.138525 22814 raft_consensus.cc:374] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:44.138550 22814 raft_consensus.cc:729] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff27111f6274a398ad71c3850fa0187, State: Initialized, Role: FOLLOWER
I20260812 03:52:44.138689 22814 consensus_queue.cc:260] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.138967 22814 ts_tablet_manager.cc:1428] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:52:44.139031 22795 heartbeater.cc:503] Master 127.17.54.254:34799 was elected leader, sending a full tablet report...
I20260812 03:52:44.140286 22816 raft_consensus.cc:348] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.140460 22816 raft_consensus.cc:374] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:44.140508 22816 raft_consensus.cc:729] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f2ea883f8746aab90c6d02a31cf1da, State: Initialized, Role: FOLLOWER
I20260812 03:52:44.140626 22816 consensus_queue.cc:260] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.140898 22816 ts_tablet_manager.cc:1428] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:52:44.140971 22500 heartbeater.cc:503] Master 127.17.54.254:34799 was elected leader, sending a full tablet report...
I20260812 03:52:44.141364 22815 tablet_bootstrap.cc:492] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f: No bootstrap required, opened a new log
I20260812 03:52:44.141453 22815 ts_tablet_manager.cc:1397] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:52:44.143190 22815 raft_consensus.cc:348] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.143321 22815 raft_consensus.cc:374] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:44.143342 22815 raft_consensus.cc:729] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edc03ebfe7414468800e443de134ad5f, State: Initialized, Role: FOLLOWER
I20260812 03:52:44.143456 22815 consensus_queue.cc:260] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.143752 22815 ts_tablet_manager.cc:1428] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:52:44.143774 22650 heartbeater.cc:503] Master 127.17.54.254:34799 was elected leader, sending a full tablet report...
W20260812 03:52:44.168936 22651 tablet.cc:2369] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:52:44.240918 22501 tablet.cc:2369] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:44.323252 22822 raft_consensus.cc:482] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:44.323391 22822 raft_consensus.cc:504] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.323812 22822 leader_election.cc:288] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6f2ea883f8746aab90c6d02a31cf1da (127.17.54.193:33707), edc03ebfe7414468800e443de134ad5f (127.17.54.194:35339)
I20260812 03:52:44.328112 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.324157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:54855 (local address 127.17.54.194:35339)
0812 03:52:44.324231 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.324233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.324243 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:44.324276 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.324276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.324286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.324388 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:44.324590 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.325384 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.327067 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.327664 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.327744 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.327753 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.327995 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.328037 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:52:44.328111 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.324169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:56587 (local address 127.17.54.193:33707)
0812 03:52:44.324231 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.324233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.324343 (+ 110us) server_negotiation.cc:407] Connection header received
0812 03:52:44.324414 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.324415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.324426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.324484 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:44.324632 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.325387 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.327067 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.327654 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.327744 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.327753 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.327995 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.328037 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:52:44.328135 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.324070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:35339 (local address 127.17.54.195:54855)
0812 03:52:44.324135 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.324148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.324199 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.324405 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.324405 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.324407 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:44.324503 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.324505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.325500 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.325501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.326945 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.326949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.327763 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.327764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.327897 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.327904 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.327974 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.328025 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":138}
I20260812 03:52:44.328142 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.324075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33707 (local address 127.17.54.195:56587)
0812 03:52:44.324278 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.324285 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.324333 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.324495 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.324496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.324497 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:44.324560 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.324562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.325500 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.325501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.326945 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:44.326949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.327763 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.327764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:44.327897 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.327904 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.327974 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.328025 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":74,"thread_start_us":103,"threads_started":1}
I20260812 03:52:44.328588 22596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0c91405f284067850e04ff133e96e8" candidate_uuid: "aff27111f6274a398ad71c3850fa0187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edc03ebfe7414468800e443de134ad5f" is_pre_election: true
I20260812 03:52:44.328670 22447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0c91405f284067850e04ff133e96e8" candidate_uuid: "aff27111f6274a398ad71c3850fa0187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" is_pre_election: true
I20260812 03:52:44.328766 22596 raft_consensus.cc:2454] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aff27111f6274a398ad71c3850fa0187 in term 0.
I20260812 03:52:44.328858 22447 raft_consensus.cc:2454] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aff27111f6274a398ad71c3850fa0187 in term 0.
I20260812 03:52:44.329079 22676 leader_election.cc:302] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aff27111f6274a398ad71c3850fa0187, edc03ebfe7414468800e443de134ad5f; no voters:
I20260812 03:52:44.329245 22822 raft_consensus.cc:2781] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:44.329326 22822 raft_consensus.cc:482] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:44.329368 22822 raft_consensus.cc:3037] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:44.330248 22822 raft_consensus.cc:504] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.330510 22822 leader_election.cc:288] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [CANDIDATE]: Term 1 election: Requested vote from peers a6f2ea883f8746aab90c6d02a31cf1da (127.17.54.193:33707), edc03ebfe7414468800e443de134ad5f (127.17.54.194:35339)
I20260812 03:52:44.330698 22596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0c91405f284067850e04ff133e96e8" candidate_uuid: "aff27111f6274a398ad71c3850fa0187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edc03ebfe7414468800e443de134ad5f"
I20260812 03:52:44.330775 22447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0c91405f284067850e04ff133e96e8" candidate_uuid: "aff27111f6274a398ad71c3850fa0187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6f2ea883f8746aab90c6d02a31cf1da"
I20260812 03:52:44.330802 22596 raft_consensus.cc:3037] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:44.330875 22447 raft_consensus.cc:3037] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:44.331599 22596 raft_consensus.cc:2454] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aff27111f6274a398ad71c3850fa0187 in term 1.
I20260812 03:52:44.331712 22447 raft_consensus.cc:2454] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aff27111f6274a398ad71c3850fa0187 in term 1.
I20260812 03:52:44.331772 22676 leader_election.cc:302] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aff27111f6274a398ad71c3850fa0187, edc03ebfe7414468800e443de134ad5f; no voters:
I20260812 03:52:44.331864 22822 raft_consensus.cc:2781] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:44.332043 22822 raft_consensus.cc:686] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 1 LEADER]: Becoming Leader. State: Replica: aff27111f6274a398ad71c3850fa0187, State: Running, Role: LEADER
I20260812 03:52:44.332149 22822 consensus_queue.cc:237] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } }
I20260812 03:52:44.333076 22279 catalog_manager.cc:5495] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 reported cstate change: term changed from 0 to 1, leader changed from <none> to aff27111f6274a398ad71c3850fa0187 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "aff27111f6274a398ad71c3850fa0187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aff27111f6274a398ad71c3850fa0187" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45511 } health_report { overall_health: HEALTHY } } }
W20260812 03:52:44.335986 22796 tablet.cc:2369] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:44.352360 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.349793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.17.54.195:45511)
0812 03:52:44.349854 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.349856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.349864 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:44.349908 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.349909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.349926 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.350004 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:44.350173 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.350736 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.351509 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:44.351708 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:44.351782 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:44.351980 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:44.352109 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:44.352202 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.352221 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.352242 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:52:44.354473 22447 raft_consensus.cc:1261] T fc0c91405f284067850e04ff133e96e8 P a6f2ea883f8746aab90c6d02a31cf1da [term 1 FOLLOWER]: Refusing update from remote peer aff27111f6274a398ad71c3850fa0187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:44.354521 22596 raft_consensus.cc:1261] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 1 FOLLOWER]: Refusing update from remote peer aff27111f6274a398ad71c3850fa0187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:44.354897 22827 consensus_queue.cc:1035] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f2ea883f8746aab90c6d02a31cf1da" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:44.355077 22822 consensus_queue.cc:1035] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edc03ebfe7414468800e443de134ad5f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:44.357810 22831 mvcc.cc:205] Tried to move back new op lower bound from 7317531706792775680 to 7317531706705199104. Current Snapshot: MvccSnapshot[applied={T|T < 7317531706792775680 or (T in {7317531706792775680})}]
I20260812 03:52:44.357932 22833 mvcc.cc:205] Tried to move back new op lower bound from 7317531706792775680 to 7317531706705199104. Current Snapshot: MvccSnapshot[applied={T|T < 7317531706792775680}]
I20260812 03:52:44.358166 22832 mvcc.cc:205] Tried to move back new op lower bound from 7317531706792775680 to 7317531706705199104. Current Snapshot: MvccSnapshot[applied={T|T < 7317531706792775680}]
I20260812 03:52:44.359848 22278 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:44.359973 22278 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:44.360860 22278 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c13ca25eb0234ed38bf5efdd5fd10112: Sending DeleteTablet for 3 replicas of tablet fc0c91405f284067850e04ff133e96e8
I20260812 03:52:44.361232 22574 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0c91405f284067850e04ff133e96e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:44 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:38168
I20260812 03:52:44.361248 22725 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0c91405f284067850e04ff133e96e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:44 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:58948
I20260812 03:52:44.361249 22425 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0c91405f284067850e04ff133e96e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:44 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:57450
I20260812 03:52:44.361498 22844 tablet_replica.cc:330] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f: stopping tablet replica
I20260812 03:52:44.361651 22844 raft_consensus.cc:2229] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:44.361773 22844 raft_consensus.cc:2258] T fc0c91405f284067850e04ff133e96e8 P edc03ebfe7414468800e443de134ad5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:44.361848 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 22353
I20260812 03:52:44.364236 22846 tablet_replica.cc:330] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: stopping tablet replica
I20260812 03:52:44.364408 22846 raft_consensus.cc:2229] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:44.364513 22846 raft_consensus.cc:2258] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:44.365029 22846 ts_tablet_manager.cc:1905] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:44.366852 22846 ts_tablet_manager.cc:1918] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:44.366943 22846 log.cc:1196] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/fc0c91405f284067850e04ff133e96e8
I20260812 03:52:44.367280 22846 ts_tablet_manager.cc:1939] T fc0c91405f284067850e04ff133e96e8 P aff27111f6274a398ad71c3850fa0187: Deleting consensus metadata
I20260812 03:52:44.367871 22263 catalog_manager.cc:4841] TS aff27111f6274a398ad71c3850fa0187 (127.17.54.195:45511): tablet fc0c91405f284067850e04ff133e96e8 (table test-table [id=a5c33da7693246b4ba991c7ba8f0a0d9]) successfully deleted
W20260812 03:52:44.370000 22262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:33707 (error 108)
I20260812 03:52:44.370625 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.370471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33707 (local address 127.0.0.1:57454)
0812 03:52:44.370539 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.370587 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 03:52:44.370676 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 22505
W20260812 03:52:44.374763 22262 catalog_manager.cc:4584] TS a6f2ea883f8746aab90c6d02a31cf1da (127.17.54.193:33707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc0c91405f284067850e04ff133e96e8: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33707: connect: Connection refused (error 111)
I20260812 03:52:44.377820 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.377678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:35339 (local address 127.0.0.1:38170)
0812 03:52:44.377742 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.377776 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.194:35339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:52:44.377919 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 22655
I20260812 03:52:44.386279 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 22242
2026-08-12T03:52:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2255 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T03:52:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:44Z Disabled control of system clock
I20260812 03:52:44.417677 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:44399
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:44399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:52:44.501776 22852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:44.502076 22852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:44.502112 22852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:44.503286 22852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:44.503332 22852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:44.503345 22852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:44.503358 22852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:44.503371 22852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:44.504741 22852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:44399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:44399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:44.504961 22852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:44.505211 22852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:44.510414 22859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:44.510371 22863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:44.510368 22860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:44.510425 22852 server_base.cc:1034] running on GCE node
I20260812 03:52:44.511060 22852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:44.511405 22852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:44.512631 22852 hybrid_clock.cc:648] HybridClock initialized: now 1786506764512619 us; error 32 us; skew 500 ppm
I20260812 03:52:44.514036 22852 webserver.cc:460] Webserver started at http://127.17.54.254:34563/ using document root <none> and password file <none>
I20260812 03:52:44.514281 22852 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:44.514402 22852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:44.514555 22852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:44.515655 22852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "6c2da9bdbe524089aa4897626718995d"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "449d0b1670aece4547cc18962bb256bc"
server_key_iv: "9cda700d24139e00506bd476be6b204c"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.516032 22852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "6c2da9bdbe524089aa4897626718995d"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "449d0b1670aece4547cc18962bb256bc"
server_key_iv: "9cda700d24139e00506bd476be6b204c"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.517392 22852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:44.518142 22869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.518366 22852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:44.518468 22852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "6c2da9bdbe524089aa4897626718995d"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "449d0b1670aece4547cc18962bb256bc"
server_key_iv: "9cda700d24139e00506bd476be6b204c"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.518549 22852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:44.521973 22852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:44.522264 22852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:44.522425 22852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:44.526428 22852 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:44399
I20260812 03:52:44.526465 22928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:44399 every 8 connection(s)
I20260812 03:52:44.526796 22852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:44.527456 22929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:44.530344 22929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d: Bootstrap starting.
I20260812 03:52:44.530944 22929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:44.531261 22929 log.cc:824] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:44.531930 22929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d: No bootstrap required, opened a new log
I20260812 03:52:44.532825 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 22852
I20260812 03:52:44.532917 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:44.533123 17627 external_mini_cluster.cc:1428] Setting key 6eb7213c5a84e46f6de632bc01987c96
I20260812 03:52:44.533795 22929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2da9bdbe524089aa4897626718995d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44399 } }
I20260812 03:52:44.533964 22929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:44.534017 22929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c2da9bdbe524089aa4897626718995d, State: Initialized, Role: FOLLOWER
I20260812 03:52:44.534147 22929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2da9bdbe524089aa4897626718995d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44399 } }
I20260812 03:52:44.534233 22929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:44.534294 22929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:44.534380 22929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:44.535108 22929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2da9bdbe524089aa4897626718995d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44399 } }
I20260812 03:52:44.535226 22929 leader_election.cc:302] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c2da9bdbe524089aa4897626718995d; no voters:
I20260812 03:52:44.535396 22929 leader_election.cc:288] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:44.535465 22935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:44.535609 22935 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [term 1 LEADER]: Becoming Leader. State: Replica: 6c2da9bdbe524089aa4897626718995d, State: Running, Role: LEADER
I20260812 03:52:44.535677 22929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:44.535775 22935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2da9bdbe524089aa4897626718995d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44399 } }
I20260812 03:52:44.535818 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.533679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39682 (local address 127.17.54.254:44399)
0812 03:52:44.533938 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.533945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.533971 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:52:44.534100 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.534108 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.534151 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.534291 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.535092 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.535095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.535096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.535100 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.535148 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.535159 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.535232 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.535300 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.535359 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260812 03:52:44.536170 22935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c2da9bdbe524089aa4897626718995d. Latest consensus state: current_term: 1 leader_uuid: "6c2da9bdbe524089aa4897626718995d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2da9bdbe524089aa4897626718995d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44399 } } }
I20260812 03:52:44.536247 22935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:44.537236 22943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:44.537328 22936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c2da9bdbe524089aa4897626718995d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2da9bdbe524089aa4897626718995d" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44399 } } }
I20260812 03:52:44.537439 22936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:44.537781 22943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:44.539354 22943 catalog_manager.cc:1347] Generated new cluster ID: 4cbcbf41fd53450d99eaa7db41dff381
I20260812 03:52:44.539417 22943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:44.547329 22943 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:44.547513 22943 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:44.551057 22943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d: Generated new TSK 0
I20260812 03:52:44.551254 22943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:44.554719 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44399
--builtin_ntp_servers=127.17.54.212:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:52:44.642766 22955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:44.642982 22955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:44.643006 22955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:44.644248 22955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:44.644297 22955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:44.644325 22955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:44.645897 22955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:44.646163 22955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:44.646520 22955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:44.652801 22963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:44.652788 22961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:44.653142 22955 server_base.cc:1034] running on GCE node
W20260812 03:52:44.652786 22965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:44.653499 22955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:44.653779 22955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:44.654944 22955 hybrid_clock.cc:648] HybridClock initialized: now 1786506764654930 us; error 31 us; skew 500 ppm
I20260812 03:52:44.656450 22955 webserver.cc:460] Webserver started at http://127.17.54.193:33989/ using document root <none> and password file <none>
I20260812 03:52:44.656728 22955 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:44.656800 22955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:44.656950 22955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:44.658061 22955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "5108aa88924c48dd9db0c54444ea3ba4"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "accbd3deebd781535a0fbc4717f4553c"
server_key_iv: "af1b2db77b149b10054dc396ba45e78f"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.658486 22955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "5108aa88924c48dd9db0c54444ea3ba4"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "accbd3deebd781535a0fbc4717f4553c"
server_key_iv: "af1b2db77b149b10054dc396ba45e78f"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.659904 22955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:44.660712 22972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.660950 22955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:44.661059 22955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "5108aa88924c48dd9db0c54444ea3ba4"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "accbd3deebd781535a0fbc4717f4553c"
server_key_iv: "af1b2db77b149b10054dc396ba45e78f"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.661139 22955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:44.666227 22955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:44.666545 22955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:44.666675 22955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:44.666887 22955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:44.667230 22955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:44.667271 22955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.667299 22955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:44.667316 22955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.674741 22955 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:33749
I20260812 03:52:44.674779 23098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:33749 every 8 connection(s)
I20260812 03:52:44.675217 22955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:44.678414 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.675972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:60497 (local address 127.17.54.254:44399)
0812 03:52:44.676050 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.676052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.676280 (+ 228us) server_negotiation.cc:407] Connection header received
0812 03:52:44.676826 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.676828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.676835 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.676898 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.678117 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.678119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.678119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.678123 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.678141 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.678146 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.678199 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.678251 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.678263 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:52:44.678560 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.675952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:44399 (local address 127.17.54.193:60497)
0812 03:52:44.676160 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.676196 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.676684 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.676954 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.676958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.676966 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.677964 (+ 998us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:52:44.677965 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.677995 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:52:44.678025 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.678195 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.678198 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.678200 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.678251 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.678355 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260812 03:52:44.679134 23099 heartbeater.cc:347] Connected to a master server at 127.17.54.254:44399
I20260812 03:52:44.679243 23099 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:44.679435 23099 heartbeater.cc:511] Master 127.17.54.254:44399 requested a full tablet report, sending...
I20260812 03:52:44.679955 22888 ts_manager.cc:194] Registered new tserver with Master: 5108aa88924c48dd9db0c54444ea3ba4 (127.17.54.193:33749)
I20260812 03:52:44.681010 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 22955
I20260812 03:52:44.681135 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:44.681326 17627 external_mini_cluster.cc:1428] Setting key 86e1f9f4c1fdab797025966d3dde7f16
I20260812 03:52:44.682183 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44399
--builtin_ntp_servers=127.17.54.212:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:52:44.766821 23102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:44.767046 23102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:44.767071 23102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:44.768294 23102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:44.768337 23102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:44.768363 23102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:44.769760 23102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:44.770031 23102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:44.770360 23102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:44.775817 23109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:44.775888 23102 server_base.cc:1034] running on GCE node
W20260812 03:52:44.775820 23108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:44.775856 23111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:44.777285 23102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:44.777539 23102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:44.778719 23102 hybrid_clock.cc:648] HybridClock initialized: now 1786506764778705 us; error 50 us; skew 500 ppm
I20260812 03:52:44.780197 23102 webserver.cc:460] Webserver started at http://127.17.54.194:42949/ using document root <none> and password file <none>
I20260812 03:52:44.780450 23102 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:44.780521 23102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:44.780696 23102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:44.781838 23102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "dca437ad8bad472985f9d44fb3ff1d0b"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "bc2af7622b26e1c3f35565e8c67adb53"
server_key_iv: "045ab6900f4599de1bf0463b3a6b6fbd"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.782275 23102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "dca437ad8bad472985f9d44fb3ff1d0b"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "bc2af7622b26e1c3f35565e8c67adb53"
server_key_iv: "045ab6900f4599de1bf0463b3a6b6fbd"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.783735 23102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:44.784528 23117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.784695 23102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:44.784798 23102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "dca437ad8bad472985f9d44fb3ff1d0b"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "bc2af7622b26e1c3f35565e8c67adb53"
server_key_iv: "045ab6900f4599de1bf0463b3a6b6fbd"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.784901 23102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:44.788515 23102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:44.788791 23102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:44.788937 23102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:44.789158 23102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:44.789491 23102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:44.789543 23102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.789583 23102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:44.789611 23102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.795795 23102 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:39857
I20260812 03:52:44.795838 23243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:39857 every 8 connection(s)
I20260812 03:52:44.796208 23102 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:52:44.797636 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 23102
I20260812 03:52:44.797724 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:44.798038 17627 external_mini_cluster.cc:1428] Setting key 9600dd48010ccbe9d97f4fc2ec50f179
I20260812 03:52:44.799124 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44399
--builtin_ntp_servers=127.17.54.212:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 03:52:44.799811 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.797055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:57779 (local address 127.17.54.254:44399)
0812 03:52:44.797191 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.797194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.797416 (+ 222us) server_negotiation.cc:407] Connection header received
0812 03:52:44.798061 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.798062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.798069 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.798126 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.799535 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.799537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.799537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.799542 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.799560 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.799565 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.799638 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.799720 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.799737 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:52:44.800096 23246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.797039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:44399 (local address 127.17.54.194:57779)
0812 03:52:44.797294 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.797331 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.797929 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.798204 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.798209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.798219 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.799376 (+ 1157us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:52:44.799377 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.799413 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:52:44.799444 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.799637 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.799644 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.799647 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.799732 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.799797 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":106,"threads_started":1}
I20260812 03:52:44.800578 23244 heartbeater.cc:347] Connected to a master server at 127.17.54.254:44399
I20260812 03:52:44.800689 23244 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:44.800855 23244 heartbeater.cc:511] Master 127.17.54.254:44399 requested a full tablet report, sending...
I20260812 03:52:44.801249 22888 ts_manager.cc:194] Registered new tserver with Master: dca437ad8bad472985f9d44fb3ff1d0b (127.17.54.194:39857)
W20260812 03:52:44.884714 23247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:44.884953 23247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:44.884992 23247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:44.886201 23247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:44.886265 23247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:44.886310 23247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:44.887835 23247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:44.888108 23247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:44.888417 23247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:52:44.893818 23247 server_base.cc:1034] running on GCE node
W20260812 03:52:44.893762 23257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:44.893762 23254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:44.893739 23255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:44.894181 23247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:44.894520 23247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:44.895709 23247 hybrid_clock.cc:648] HybridClock initialized: now 1786506764895694 us; error 33 us; skew 500 ppm
I20260812 03:52:44.897028 23247 webserver.cc:460] Webserver started at http://127.17.54.195:34223/ using document root <none> and password file <none>
I20260812 03:52:44.897318 23247 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:44.897368 23247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:44.897529 23247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:44.898748 23247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "13302b453fcac7208ca4ef568ba9d947"
server_key_iv: "3bc338bc48d215b405bc2668b0a22e1c"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.899127 23247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "13302b453fcac7208ca4ef568ba9d947"
server_key_iv: "3bc338bc48d215b405bc2668b0a22e1c"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.900494 23247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:52:44.901275 23265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.901440 23247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:44.901540 23247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984"
format_stamp: "Formatted at 2026-08-12 03:52:44 on dist-test-slave-tm1g"
server_key: "13302b453fcac7208ca4ef568ba9d947"
server_key_iv: "3bc338bc48d215b405bc2668b0a22e1c"
server_key_version: "encryptionkey@0"
I20260812 03:52:44.901618 23247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:44.905179 23247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:44.905474 23247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:44.905614 23247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:44.905831 23247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:44.906147 23247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:44.906199 23247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.906239 23247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:44.906267 23247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:44.912312 23247 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:45111
I20260812 03:52:44.912348 23390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:45111 every 8 connection(s)
I20260812 03:52:44.912680 23247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:44.915022 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 23247
I20260812 03:52:44.915109 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:44.915310 17627 external_mini_cluster.cc:1428] Setting key 391a016f15e0ed0aa68ec57ca183f36d
I20260812 03:52:44.916038 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.913487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:42435 (local address 127.17.54.254:44399)
0812 03:52:44.913541 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.913544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.913711 (+ 167us) server_negotiation.cc:407] Connection header received
0812 03:52:44.914218 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.914219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.914225 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.914294 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.915772 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.915773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.915774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.915778 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.915795 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.915799 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.915864 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.915944 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.915963 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:52:44.916231 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.913370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:44399 (local address 127.17.54.195:42435)
0812 03:52:44.913605 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.913640 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.914108 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.914468 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.914472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.914481 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.915627 (+ 1146us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:52:44.915627 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.915668 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:52:44.915697 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.915853 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.915857 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.915860 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.915946 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.916011 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":111,"threads_started":1}
I20260812 03:52:44.916712 23391 heartbeater.cc:347] Connected to a master server at 127.17.54.254:44399
I20260812 03:52:44.916796 23391 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:44.916918 23391 heartbeater.cc:511] Master 127.17.54.254:44399 requested a full tablet report, sending...
I20260812 03:52:44.917244 22888 ts_manager.cc:194] Registered new tserver with Master: 87a7b7e7fd9c4eeaa0ccae8c57134984 (127.17.54.195:45111)
I20260812 03:52:44.927599 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:52:44.930374 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.929059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39688 (local address 127.17.54.254:44399)
0812 03:52:44.929158 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.929160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.929255 (+ 95us) server_negotiation.cc:407] Connection header received
0812 03:52:44.929377 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.929378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.929384 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.929441 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.930099 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.930100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.930100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.930105 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.930123 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.930127 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.930191 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.930252 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.930269 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:52:44.932222 22888 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:44.933826 22888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:44.938323 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.936431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.17.54.195:45111)
0812 03:52:44.937136 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.937141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.937150 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:44.937177 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.937179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.937189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.937469 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.937981 (+ 512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.937988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.937989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.937993 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.938023 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.938043 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.938103 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.938189 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.938205 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":86,"threads_started":1}
I20260812 03:52:44.938295 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.936387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:39857 (local address 127.0.0.1:45844)
0812 03:52:44.936779 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.936789 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.936836 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.937471 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.937472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.937474 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.937881 (+ 407us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:52:44.937881 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.937889 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:52:44.937901 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.938130 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.938130 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.938131 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.938184 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.938199 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260812 03:52:44.938342 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.936339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:45111 (local address 127.0.0.1:34482)
0812 03:52:44.936518 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.936558 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.936619 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.937307 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.937308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.937311 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.937737 (+ 426us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:52:44.937738 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.937754 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:52:44.937781 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.938099 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.938106 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.938109 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.938181 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.938197 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":74,"threads_started":1}
I20260812 03:52:44.938548 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.936882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.17.54.194:39857)
0812 03:52:44.937232 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.937236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.937246 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:44.937272 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.937275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.937284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.937615 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.937973 (+ 358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.937979 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.937979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.937983 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.938020 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.938044 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.938102 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.938430 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.938449 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":103,"threads_started":1}
I20260812 03:52:44.938721 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.936386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33749 (local address 127.0.0.1:35966)
0812 03:52:44.936886 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.936895 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.936998 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.937159 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.937166 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.937173 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.938043 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:52:44.938044 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.938050 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:52:44.938061 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.938350 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.938351 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.938352 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.938656 (+ 304us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.938668 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":99,"threads_started":1}
I20260812 03:52:44.938720 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.936638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.17.54.193:33749)
0812 03:52:44.936855 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.936859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.936975 (+ 116us) server_negotiation.cc:407] Connection header received
0812 03:52:44.937062 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.937068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.937079 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.937159 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.938153 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.938161 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.938161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.938167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.938204 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.938235 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.938290 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.938605 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.938629 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":122,"threads_started":1}
I20260812 03:52:44.939244 23173 tablet_service.cc:1467] Processing CreateTablet for tablet 302ae2473deb4245986e004d8241f3b5 (DEFAULT_TABLE table=test-table [id=7c91aa82413144ffa4f91955c611d481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:44.939493 23320 tablet_service.cc:1467] Processing CreateTablet for tablet 302ae2473deb4245986e004d8241f3b5 (DEFAULT_TABLE table=test-table [id=7c91aa82413144ffa4f91955c611d481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:44.939666 23173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302ae2473deb4245986e004d8241f3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:44.939841 23320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302ae2473deb4245986e004d8241f3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:44.942468 23407 tablet_bootstrap.cc:492] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: Bootstrap starting.
I20260812 03:52:44.942818 23408 tablet_bootstrap.cc:492] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: Bootstrap starting.
I20260812 03:52:44.942780 23027 tablet_service.cc:1467] Processing CreateTablet for tablet 302ae2473deb4245986e004d8241f3b5 (DEFAULT_TABLE table=test-table [id=7c91aa82413144ffa4f91955c611d481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:44.943123 23027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302ae2473deb4245986e004d8241f3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:44.943373 23407 tablet_bootstrap.cc:654] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:44.943584 23408 tablet_bootstrap.cc:654] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:44.943706 23407 log.cc:824] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:44.943879 23408 log.cc:824] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:44.944604 23407 tablet_bootstrap.cc:492] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: No bootstrap required, opened a new log
I20260812 03:52:44.944664 23407 ts_tablet_manager.cc:1397] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:52:44.945865 23414 tablet_bootstrap.cc:492] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4: Bootstrap starting.
I20260812 03:52:44.946633 23408 tablet_bootstrap.cc:492] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: No bootstrap required, opened a new log
I20260812 03:52:44.946717 23408 ts_tablet_manager.cc:1397] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:52:44.946888 23414 tablet_bootstrap.cc:654] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:44.946995 23407 raft_consensus.cc:348] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.947185 23407 raft_consensus.cc:374] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:44.947232 23407 raft_consensus.cc:729] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a7b7e7fd9c4eeaa0ccae8c57134984, State: Initialized, Role: FOLLOWER
I20260812 03:52:44.947230 23414 log.cc:824] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:44.947391 23407 consensus_queue.cc:260] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.947676 23407 ts_tablet_manager.cc:1428] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:52:44.947728 23391 heartbeater.cc:503] Master 127.17.54.254:44399 was elected leader, sending a full tablet report...
I20260812 03:52:44.948226 23414 tablet_bootstrap.cc:492] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4: No bootstrap required, opened a new log
I20260812 03:52:44.948271 23414 ts_tablet_manager.cc:1397] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:44.949023 23408 raft_consensus.cc:348] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.949134 23408 raft_consensus.cc:374] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:44.949154 23408 raft_consensus.cc:729] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca437ad8bad472985f9d44fb3ff1d0b, State: Initialized, Role: FOLLOWER
I20260812 03:52:44.949235 23408 consensus_queue.cc:260] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.949417 23408 ts_tablet_manager.cc:1428] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:52:44.949522 23244 heartbeater.cc:503] Master 127.17.54.254:44399 was elected leader, sending a full tablet report...
I20260812 03:52:44.950361 23414 raft_consensus.cc:348] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.950515 23414 raft_consensus.cc:374] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:44.950567 23414 raft_consensus.cc:729] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5108aa88924c48dd9db0c54444ea3ba4, State: Initialized, Role: FOLLOWER
I20260812 03:52:44.950677 23414 consensus_queue.cc:260] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.950950 23414 ts_tablet_manager.cc:1428] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:52:44.951071 23099 heartbeater.cc:503] Master 127.17.54.254:44399 was elected leader, sending a full tablet report...
I20260812 03:52:44.955621 23417 raft_consensus.cc:482] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:44.955714 23417 raft_consensus.cc:504] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.956007 23417 leader_election.cc:288] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5108aa88924c48dd9db0c54444ea3ba4 (127.17.54.193:33749), dca437ad8bad472985f9d44fb3ff1d0b (127.17.54.194:39857)
I20260812 03:52:44.957325 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.956157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:39857 (local address 127.17.54.195:35719)
0812 03:52:44.956202 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.956214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.956256 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.956446 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.956447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.956450 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.956923 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:52:44.956924 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.956932 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:52:44.956946 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.957183 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.957184 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.957185 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.957240 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.957256 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 03:52:44.957413 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.956280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:35719 (local address 127.17.54.194:39857)
0812 03:52:44.956318 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.956320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.956329 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:44.956353 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.956354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.956360 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.956433 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.957062 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.957064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.957064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.957070 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.957091 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.957096 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.957178 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.957255 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.957271 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:52:44.957584 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.956194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:46925 (local address 127.17.54.193:33749)
0812 03:52:44.956241 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.956243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.956397 (+ 154us) server_negotiation.cc:407] Connection header received
0812 03:52:44.956448 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.956449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.956455 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.956539 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.957270 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.957271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.957271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.957276 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.957293 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.957297 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.957333 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.957479 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.957497 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:44.957582 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.956160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33749 (local address 127.17.54.195:46925)
0812 03:52:44.956320 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.956332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:44.956392 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:44.956507 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:44.956508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:44.956510 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:44.957109 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:52:44.957110 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:44.957117 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:52:44.957130 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:44.957385 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:44.957386 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:44.957387 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:52:44.957466 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:52:44.957481 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":23,"thread_start_us":88,"threads_started":1}
I20260812 03:52:44.957700 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302ae2473deb4245986e004d8241f3b5" candidate_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" is_pre_election: true
I20260812 03:52:44.957841 23196 raft_consensus.cc:2454] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a7b7e7fd9c4eeaa0ccae8c57134984 in term 0.
I20260812 03:52:44.957921 23049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302ae2473deb4245986e004d8241f3b5" candidate_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5108aa88924c48dd9db0c54444ea3ba4" is_pre_election: true
I20260812 03:52:44.958052 23049 raft_consensus.cc:2454] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a7b7e7fd9c4eeaa0ccae8c57134984 in term 0.
I20260812 03:52:44.958087 23271 leader_election.cc:302] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87a7b7e7fd9c4eeaa0ccae8c57134984, dca437ad8bad472985f9d44fb3ff1d0b; no voters:
I20260812 03:52:44.958225 23417 raft_consensus.cc:2781] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:44.958266 23417 raft_consensus.cc:482] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:44.958289 23417 raft_consensus.cc:3037] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:44.959064 23417 raft_consensus.cc:504] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.959208 23417 leader_election.cc:288] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [CANDIDATE]: Term 1 election: Requested vote from peers 5108aa88924c48dd9db0c54444ea3ba4 (127.17.54.193:33749), dca437ad8bad472985f9d44fb3ff1d0b (127.17.54.194:39857)
I20260812 03:52:44.959344 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302ae2473deb4245986e004d8241f3b5" candidate_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dca437ad8bad472985f9d44fb3ff1d0b"
I20260812 03:52:44.959383 23049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302ae2473deb4245986e004d8241f3b5" candidate_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5108aa88924c48dd9db0c54444ea3ba4"
I20260812 03:52:44.959416 23196 raft_consensus.cc:3037] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:44.959481 23049 raft_consensus.cc:3037] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:44.960104 23196 raft_consensus.cc:2454] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a7b7e7fd9c4eeaa0ccae8c57134984 in term 1.
I20260812 03:52:44.960259 23271 leader_election.cc:302] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87a7b7e7fd9c4eeaa0ccae8c57134984, dca437ad8bad472985f9d44fb3ff1d0b; no voters:
I20260812 03:52:44.960278 23049 raft_consensus.cc:2454] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a7b7e7fd9c4eeaa0ccae8c57134984 in term 1.
I20260812 03:52:44.960359 23417 raft_consensus.cc:2781] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:44.960516 23417 raft_consensus.cc:686] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 1 LEADER]: Becoming Leader. State: Replica: 87a7b7e7fd9c4eeaa0ccae8c57134984, State: Running, Role: LEADER
I20260812 03:52:44.960609 23417 consensus_queue.cc:237] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } }
I20260812 03:52:44.961328 22887 catalog_manager.cc:5495] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87a7b7e7fd9c4eeaa0ccae8c57134984 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a7b7e7fd9c4eeaa0ccae8c57134984" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:44.966444 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.965409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34490 (local address 127.17.54.195:45111)
0812 03:52:44.965469 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:44.965471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:44.965497 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:52:44.965549 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:44.965549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:44.965559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:44.965615 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:44.966221 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:44.966222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:44.966223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:44.966227 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:44.966242 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:44.966246 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:44.966292 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:44.966362 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 03:52:44.966376 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:52:44.968196 23049 raft_consensus.cc:1261] T 302ae2473deb4245986e004d8241f3b5 P 5108aa88924c48dd9db0c54444ea3ba4 [term 1 FOLLOWER]: Refusing update from remote peer 87a7b7e7fd9c4eeaa0ccae8c57134984: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:44.968536 23196 raft_consensus.cc:1261] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 1 FOLLOWER]: Refusing update from remote peer 87a7b7e7fd9c4eeaa0ccae8c57134984: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:44.968564 23423 consensus_queue.cc:1035] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5108aa88924c48dd9db0c54444ea3ba4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:44.968834 23423 consensus_queue.cc:1035] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dca437ad8bad472985f9d44fb3ff1d0b" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:44.971093 23426 mvcc.cc:205] Tried to move back new op lower bound from 7317531709307105280 to 7317531709279322112. Current Snapshot: MvccSnapshot[applied={T|T < 7317531709307105280}]
I20260812 03:52:44.971802 23427 mvcc.cc:205] Tried to move back new op lower bound from 7317531709307105280 to 7317531709279322112. Current Snapshot: MvccSnapshot[applied={T|T < 7317531709307105280}]
I20260812 03:52:44.972128 23428 mvcc.cc:205] Tried to move back new op lower bound from 7317531709307105280 to 7317531709279322112. Current Snapshot: MvccSnapshot[applied={T|T < 7317531709307105280}]
I20260812 03:52:44.973984 22887 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:44.974073 22887 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:44.974792 22887 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c2da9bdbe524089aa4897626718995d: Sending DeleteTablet for 3 replicas of tablet 302ae2473deb4245986e004d8241f3b5
I20260812 03:52:44.975137 23173 tablet_service.cc:1514] Processing DeleteTablet for tablet 302ae2473deb4245986e004d8241f3b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:44 UTC) from {username='slave'} at 127.0.0.1:45844
I20260812 03:52:44.975136 23027 tablet_service.cc:1514] Processing DeleteTablet for tablet 302ae2473deb4245986e004d8241f3b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:44 UTC) from {username='slave'} at 127.0.0.1:35966
I20260812 03:52:44.975162 23320 tablet_service.cc:1514] Processing DeleteTablet for tablet 302ae2473deb4245986e004d8241f3b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:44 UTC) from {username='slave'} at 127.0.0.1:34482
I20260812 03:52:44.975430 23436 tablet_replica.cc:330] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: stopping tablet replica
I20260812 03:52:44.975531 23436 raft_consensus.cc:2229] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:44.975615 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 22955
I20260812 03:52:44.977890 23437 tablet_replica.cc:330] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: stopping tablet replica
I20260812 03:52:44.977982 23437 raft_consensus.cc:2229] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:44.978094 23436 raft_consensus.cc:2258] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:44.978101 23437 raft_consensus.cc:2258] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:44.978586 23437 ts_tablet_manager.cc:1905] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:44.978744 23436 ts_tablet_manager.cc:1905] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:44.980774 23436 ts_tablet_manager.cc:1918] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:44.980783 23437 ts_tablet_manager.cc:1918] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:44.980852 23437 log.cc:1196] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/302ae2473deb4245986e004d8241f3b5
I20260812 03:52:44.980856 23436 log.cc:1196] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/302ae2473deb4245986e004d8241f3b5
I20260812 03:52:44.981134 23437 ts_tablet_manager.cc:1939] T 302ae2473deb4245986e004d8241f3b5 P 87a7b7e7fd9c4eeaa0ccae8c57134984: Deleting consensus metadata
I20260812 03:52:44.981161 23436 ts_tablet_manager.cc:1939] T 302ae2473deb4245986e004d8241f3b5 P dca437ad8bad472985f9d44fb3ff1d0b: Deleting consensus metadata
I20260812 03:52:44.981554 22870 catalog_manager.cc:4841] TS 87a7b7e7fd9c4eeaa0ccae8c57134984 (127.17.54.195:45111): tablet 302ae2473deb4245986e004d8241f3b5 (table test-table [id=7c91aa82413144ffa4f91955c611d481]) successfully deleted
I20260812 03:52:44.981638 22873 catalog_manager.cc:4841] TS dca437ad8bad472985f9d44fb3ff1d0b (127.17.54.194:39857): tablet 302ae2473deb4245986e004d8241f3b5 (table test-table [id=7c91aa82413144ffa4f91955c611d481]) successfully deleted
W20260812 03:52:44.984083 22871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:33749 (error 108)
I20260812 03:52:44.984663 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:44.984518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33749 (local address 127.0.0.1:35974)
0812 03:52:44.984578 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:44.984627 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:52:44.984688 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 23102
W20260812 03:52:44.986864 22871 catalog_manager.cc:4584] TS 5108aa88924c48dd9db0c54444ea3ba4 (127.17.54.193:33749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 302ae2473deb4245986e004d8241f3b5: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33749: connect: Connection refused (error 111)
I20260812 03:52:44.995188 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 23247
I20260812 03:52:45.002777 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 22852
2026-08-12T03:52:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (617 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 03:52:45.027081 17627 mini_oidc.cc:152] Starting JWKS server
I20260812 03:52:45.028319 17627 webserver.cc:460] Webserver started at https://127.0.0.1:35279/ using document root <none> and password file <none>
I20260812 03:52:45.028450 17627 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:52:45.028647 17627 webserver.cc:460] Webserver started at http://127.0.0.1:46669/ using document root <none> and password file <none>
2026-08-12T03:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:45Z Disabled control of system clock
I20260812 03:52:45.038674 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34285
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:34285
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35279/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/testchainca.cert with env {}
W20260812 03:52:45.123617 23446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:45.123921 23446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:45.123965 23446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:45.125360 23446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:45.125427 23446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:45.125447 23446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:45.125468 23446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:45.125499 23446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:52:45.125526 23446 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:52:45.125553 23446 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35279/jwks.json
I20260812 03:52:45.127192 23446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:34285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35279/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:45.127482 23446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:45.127705 23446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:45.132903 23451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:45.133044 23446 server_base.cc:1034] running on GCE node
W20260812 03:52:45.132959 23452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:45.132889 23454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:45.133419 23446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:45.133698 23446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:45.134837 23446 hybrid_clock.cc:648] HybridClock initialized: now 1786506765134823 us; error 38 us; skew 500 ppm
I20260812 03:52:45.136137 23446 webserver.cc:460] Webserver started at http://127.17.54.254:46185/ using document root <none> and password file <none>
I20260812 03:52:45.136382 23446 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:45.136431 23446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:45.136601 23446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:45.137732 23446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "fe2a85ba86404c9cb61ddc3cdcd82586"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "32f2c2a75c924a4cc30d1bc1e3d9e946"
server_key_iv: "bc920031d08e9e0bcc9cc3f2d294c52e"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.138108 23446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "fe2a85ba86404c9cb61ddc3cdcd82586"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "32f2c2a75c924a4cc30d1bc1e3d9e946"
server_key_iv: "bc920031d08e9e0bcc9cc3f2d294c52e"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.139554 23446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:45.140344 23462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:45.140550 23446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:45.140653 23446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "fe2a85ba86404c9cb61ddc3cdcd82586"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "32f2c2a75c924a4cc30d1bc1e3d9e946"
server_key_iv: "bc920031d08e9e0bcc9cc3f2d294c52e"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.140733 23446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:45.156143 23446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:45.156507 23446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:45.156665 23446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:45.160851 23446 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:34285
I20260812 03:52:45.160890 23519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:34285 every 8 connection(s)
I20260812 03:52:45.161270 23446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:45.162407 23520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:45.164034 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 23446
I20260812 03:52:45.164141 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:45.164343 17627 external_mini_cluster.cc:1428] Setting key 18d8e88d76b86066e92731ebc9f3c36c
I20260812 03:52:45.165335 23520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586: Bootstrap starting.
I20260812 03:52:45.166035 23520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:45.166426 23520 log.cc:824] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:45.167151 23520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586: No bootstrap required, opened a new log
I20260812 03:52:45.169546 23520 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2a85ba86404c9cb61ddc3cdcd82586" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34285 } }
I20260812 03:52:45.169721 23520 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:45.169763 23520 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2a85ba86404c9cb61ddc3cdcd82586, State: Initialized, Role: FOLLOWER
I20260812 03:52:45.169898 23520 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2a85ba86404c9cb61ddc3cdcd82586" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34285 } }
I20260812 03:52:45.170009 23520 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:45.170055 23520 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:45.170106 23520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:45.170123 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:45.164907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.17.54.254:34285)
0812 03:52:45.165108 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:45.165116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:45.165141 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:52:45.165619 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:45.165627 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:45.165690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:45.165809 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:45.166069 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.167093 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.168173 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.168590 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.168679 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.169380 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:45.169383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:45.169383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:45.169555 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:45.169590 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:45.169601 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:45.169650 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:45.169762 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 03:52:45.169846 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260812 03:52:45.170852 23520 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2a85ba86404c9cb61ddc3cdcd82586" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34285 } }
I20260812 03:52:45.170966 23520 leader_election.cc:302] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe2a85ba86404c9cb61ddc3cdcd82586; no voters:
I20260812 03:52:45.171137 23520 leader_election.cc:288] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:45.171195 23525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:45.171299 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:45.171371 23525 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [term 1 LEADER]: Becoming Leader. State: Replica: fe2a85ba86404c9cb61ddc3cdcd82586, State: Running, Role: LEADER
I20260812 03:52:45.171434 23520 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:45.171483 23525 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2a85ba86404c9cb61ddc3cdcd82586" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34285 } }
I20260812 03:52:45.171836 23525 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe2a85ba86404c9cb61ddc3cdcd82586. Latest consensus state: current_term: 1 leader_uuid: "fe2a85ba86404c9cb61ddc3cdcd82586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2a85ba86404c9cb61ddc3cdcd82586" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34285 } } }
I20260812 03:52:45.171847 23526 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe2a85ba86404c9cb61ddc3cdcd82586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2a85ba86404c9cb61ddc3cdcd82586" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34285 } } }
I20260812 03:52:45.171924 23525 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:45.171974 23526 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:45.172170 23532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:45.172726 23532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:45.174404 23532 catalog_manager.cc:1347] Generated new cluster ID: 62322573267d4d9a9c68893d845b6b90
I20260812 03:52:45.174469 23532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:45.183657 23532 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:45.184269 23532 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:45.194332 23532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe2a85ba86404c9cb61ddc3cdcd82586: Generated new TSK 0
I20260812 03:52:45.194571 23532 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:52:45.222278 23551 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 03:52:45.222491 23551 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260812 03:52:45.223059 23550 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:45.207430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34285 (local address 127.0.0.1:57492)
0812 03:52:45.207632 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:45.207642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:45.207722 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:45.207918 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:45.207919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:45.207921 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:52:45.208016 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:45.208018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.208693 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.208694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:45.209285 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:45.209287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.209680 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.209682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:45.209750 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.209769 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:52:45.222642 (+ 12873us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:52:45.222741 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.54.254:34285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.54.254:34285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 03:52:45.223404 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:45.207495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57492 (local address 127.17.54.254:34285)
0812 03:52:45.207556 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:45.207559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:45.207699 (+ 140us) server_negotiation.cc:407] Connection header received
0812 03:52:45.207824 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:45.207825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:45.207842 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:45.207910 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:52:45.208089 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.208610 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.209384 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.209596 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.209666 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.209894 (+ 228us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:52:45.222460 (+ 12566us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:52:45.223330 (+ 870us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57492: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:52:45.223413 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 23446
2026-08-12T03:52:45Z chronyd exiting
I20260812 03:52:45.261513 17627 test_util.cc:175] -----------------------------------------------
I20260812 03:52:45.261600 17627 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506741478267-17627-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 03:52:45.262436 17627 mini_oidc.cc:152] Starting JWKS server
I20260812 03:52:45.263175 17627 webserver.cc:460] Webserver started at https://127.0.0.1:39023/ using document root <none> and password file <none>
I20260812 03:52:45.263217 17627 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:52:45.263382 17627 webserver.cc:460] Webserver started at http://127.0.0.1:40173/ using document root <none> and password file <none>
2026-08-12T03:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:45Z Disabled control of system clock
I20260812 03:52:45.272428 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:32935
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:32935
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39023/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/testchainca.cert with env {}
W20260812 03:52:45.356776 23559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:45.357004 23559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:45.357028 23559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:45.358248 23559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:45.358291 23559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:45.358352 23559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:45.358378 23559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:45.358397 23559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:52:45.358423 23559 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:52:45.358451 23559 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39023/jwks.json
I20260812 03:52:45.360037 23559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:32935
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:32935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39023/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:45.360302 23559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:45.360529 23559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:45.365684 23568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:45.365691 23565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:45.365684 23571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:45.365907 23559 server_base.cc:1034] running on GCE node
I20260812 03:52:45.366255 23559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:45.366623 23559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:45.367820 23559 hybrid_clock.cc:648] HybridClock initialized: now 1786506765367808 us; error 35 us; skew 500 ppm
I20260812 03:52:45.369122 23559 webserver.cc:460] Webserver started at http://127.17.54.254:34005/ using document root <none> and password file <none>
I20260812 03:52:45.369421 23559 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:45.369498 23559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:45.369645 23559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:45.370816 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "7c6561f2815d463f9d9edf1981d12515"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "9f0914ca9051b80979ca23583b24f26d"
server_key_iv: "6e1daaa1204417668496cc1934ec1e82"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.371186 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "7c6561f2815d463f9d9edf1981d12515"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "9f0914ca9051b80979ca23583b24f26d"
server_key_iv: "6e1daaa1204417668496cc1934ec1e82"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.372624 23559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:52:45.373409 23578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:45.373586 23559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:45.373690 23559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "7c6561f2815d463f9d9edf1981d12515"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "9f0914ca9051b80979ca23583b24f26d"
server_key_iv: "6e1daaa1204417668496cc1934ec1e82"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.373765 23559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:45.402272 23559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:45.402709 23559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:45.402874 23559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:45.407110 23559 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:32935
I20260812 03:52:45.407172 23636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:32935 every 8 connection(s)
I20260812 03:52:45.407553 23559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:45.408313 23637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:45.411326 23637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515: Bootstrap starting.
I20260812 03:52:45.411957 23637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:45.412225 23637 log.cc:824] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:45.413044 23637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515: No bootstrap required, opened a new log
I20260812 03:52:45.414809 23637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6561f2815d463f9d9edf1981d12515" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 32935 } }
I20260812 03:52:45.414965 23637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:45.414990 23637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c6561f2815d463f9d9edf1981d12515, State: Initialized, Role: FOLLOWER
I20260812 03:52:45.415158 23637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6561f2815d463f9d9edf1981d12515" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 32935 } }
I20260812 03:52:45.415220 23637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:45.415257 23637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:45.415307 23637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:45.415895 23637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6561f2815d463f9d9edf1981d12515" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 32935 } }
I20260812 03:52:45.415987 23637 leader_election.cc:302] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c6561f2815d463f9d9edf1981d12515; no voters:
I20260812 03:52:45.416196 23637 leader_election.cc:288] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:45.416241 23640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:45.416483 23637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:45.416491 23640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [term 1 LEADER]: Becoming Leader. State: Replica: 7c6561f2815d463f9d9edf1981d12515, State: Running, Role: LEADER
I20260812 03:52:45.416694 23640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6561f2815d463f9d9edf1981d12515" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 32935 } }
I20260812 03:52:45.417136 23641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c6561f2815d463f9d9edf1981d12515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6561f2815d463f9d9edf1981d12515" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 32935 } } }
I20260812 03:52:45.417282 23641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:45.417599 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 23559
I20260812 03:52:45.417712 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:45.417834 23650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:45.417940 17627 external_mini_cluster.cc:1428] Setting key b5233ee0ba7b922353e00972110ed847
I20260812 03:52:45.417346 23642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c6561f2815d463f9d9edf1981d12515. Latest consensus state: current_term: 1 leader_uuid: "7c6561f2815d463f9d9edf1981d12515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6561f2815d463f9d9edf1981d12515" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 32935 } } }
I20260812 03:52:45.418273 23642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:45.418390 23650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:45.420212 23650 catalog_manager.cc:1347] Generated new cluster ID: 79ad0b7569a34070bb653434cbc86adf
I20260812 03:52:45.420267 23650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:45.424978 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:45.418541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51948 (local address 127.17.54.254:32935)
0812 03:52:45.419263 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:45.419270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:45.419295 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:52:45.419392 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:45.419397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:45.419450 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:45.419592 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:45.419809 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.421344 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.422445 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.423184 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.423341 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.424042 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:45.424046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:45.424046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:45.424279 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:45.424337 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:45.424358 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:45.424431 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:45.424506 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 03:52:45.424715 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":88,"threads_started":1}
I20260812 03:52:45.425984 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:45.433414 23650 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:45.433956 23650 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:45.443176 23650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c6561f2815d463f9d9edf1981d12515: Generated new TSK 0
I20260812 03:52:45.443415 23650 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:52:45.481254 23670 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 03:52:45.481359 23670 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:52:45.481798 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:45.456609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.17.54.254:32935)
0812 03:52:45.456660 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:45.456662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:45.456738 (+ 76us) server_negotiation.cc:407] Connection header received
0812 03:52:45.456822 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:45.456822 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:45.456833 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:45.456896 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:52:45.457097 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.457670 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.459037 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.459264 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.459325 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.459572 (+ 247us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:52:45.481583 (+ 22011us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39023/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:52:45.481739 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39023/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 03:52:45.481884 23660 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39023/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 03:52:45.481966 23669 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:45.456472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:32935 (local address 127.0.0.1:51950)
0812 03:52:45.456658 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:45.456672 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:45.456748 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:45.456920 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:45.456921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:45.456924 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:52:45.456998 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:45.457001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.457762 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.457763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:45.458921 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:45.458926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.459357 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.459364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:45.459443 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.459451 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:52:45.481716 (+ 22265us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39023/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:52:45.481857 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.54.254:32935: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39023/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
I20260812 03:52:45.482414 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 23559
2026-08-12T03:52:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (261 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 03:52:45.523862 17627 mini_oidc.cc:152] Starting JWKS server
I20260812 03:52:45.524559 17627 webserver.cc:460] Webserver started at https://127.0.0.1:35109/ using document root <none> and password file <none>
I20260812 03:52:45.524631 17627 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:52:45.524809 17627 webserver.cc:460] Webserver started at http://127.0.0.1:39877/ using document root <none> and password file <none>
2026-08-12T03:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:45Z Disabled control of system clock
I20260812 03:52:45.533653 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37487
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:37487
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35109/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:52:45.633384 23676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:45.633663 23676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:45.633708 23676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:45.634990 23676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:45.635063 23676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:45.635085 23676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:45.635106 23676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:45.635135 23676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:52:45.635159 23676 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:52:45.635190 23676 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35109/jwks.json
I20260812 03:52:45.636655 23676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:37487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35109/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:45.636966 23676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:45.637291 23676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:45.643276 23685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:45.643414 23676 server_base.cc:1034] running on GCE node
W20260812 03:52:45.643246 23682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:45.643265 23683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:45.643802 23676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:45.644173 23676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:45.645332 23676 hybrid_clock.cc:648] HybridClock initialized: now 1786506765645314 us; error 41 us; skew 500 ppm
I20260812 03:52:45.646890 23676 webserver.cc:460] Webserver started at http://127.17.54.254:41587/ using document root <none> and password file <none>
I20260812 03:52:45.647187 23676 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:45.647261 23676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:45.647426 23676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:45.648686 23676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "b457fc1f92fb443782e00dd0cf3dcf2f"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "80bbdfad858ddd716bbc189aff6c0f44"
server_key_iv: "ad58bdb853204be1af20072ba392a0ab"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.649096 23676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "b457fc1f92fb443782e00dd0cf3dcf2f"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "80bbdfad858ddd716bbc189aff6c0f44"
server_key_iv: "ad58bdb853204be1af20072ba392a0ab"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.650777 23676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260812 03:52:45.651620 23692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:45.651827 23676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:45.651935 23676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "b457fc1f92fb443782e00dd0cf3dcf2f"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "80bbdfad858ddd716bbc189aff6c0f44"
server_key_iv: "ad58bdb853204be1af20072ba392a0ab"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.652036 23676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:45.707624 23676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:45.708024 23676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:45.708201 23676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:45.712785 23676 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:37487
I20260812 03:52:45.712841 23753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:37487 every 8 connection(s)
I20260812 03:52:45.713225 23676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:45.713965 23754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:45.717025 23754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f: Bootstrap starting.
I20260812 03:52:45.718116 23754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:45.718601 23754 log.cc:824] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:45.719385 23754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f: No bootstrap required, opened a new log
I20260812 03:52:45.719671 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 23676
I20260812 03:52:45.719781 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:45.719959 17627 external_mini_cluster.cc:1428] Setting key aa91f587afa7f75b419632b0d546256e
I20260812 03:52:45.721668 23754 raft_consensus.cc:348] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b457fc1f92fb443782e00dd0cf3dcf2f" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37487 } }
I20260812 03:52:45.721841 23754 raft_consensus.cc:374] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:45.721920 23754 raft_consensus.cc:729] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b457fc1f92fb443782e00dd0cf3dcf2f, State: Initialized, Role: FOLLOWER
I20260812 03:52:45.722088 23754 consensus_queue.cc:260] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b457fc1f92fb443782e00dd0cf3dcf2f" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37487 } }
I20260812 03:52:45.722193 23754 raft_consensus.cc:388] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:45.722239 23754 raft_consensus.cc:482] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:45.722285 23754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:45.726502 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:45.720556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.17.54.254:37487)
0812 03:52:45.720790 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:45.720798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:45.720824 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:52:45.721633 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:45.721641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:45.721721 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:45.721882 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:45.722222 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.723614 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.724418 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.724877 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.725018 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.725746 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:45.725750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:45.725751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:45.725953 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:45.725991 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:45.726003 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:45.726083 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:45.726155 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:52:45.726245 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260812 03:52:45.727721 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:45.781525 23754 raft_consensus.cc:504] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b457fc1f92fb443782e00dd0cf3dcf2f" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37487 } }
I20260812 03:52:45.781720 23754 leader_election.cc:302] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b457fc1f92fb443782e00dd0cf3dcf2f; no voters:
I20260812 03:52:45.782030 23754 leader_election.cc:288] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:45.782223 23759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:45.782428 23754 sys_catalog.cc:564] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:45.782471 23759 raft_consensus.cc:686] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [term 1 LEADER]: Becoming Leader. State: Replica: b457fc1f92fb443782e00dd0cf3dcf2f, State: Running, Role: LEADER
I20260812 03:52:45.782738 23759 consensus_queue.cc:237] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b457fc1f92fb443782e00dd0cf3dcf2f" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37487 } }
I20260812 03:52:45.783218 23762 sys_catalog.cc:455] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b457fc1f92fb443782e00dd0cf3dcf2f. Latest consensus state: current_term: 1 leader_uuid: "b457fc1f92fb443782e00dd0cf3dcf2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b457fc1f92fb443782e00dd0cf3dcf2f" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37487 } } }
I20260812 03:52:45.783344 23762 sys_catalog.cc:458] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:45.783463 23761 sys_catalog.cc:455] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b457fc1f92fb443782e00dd0cf3dcf2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b457fc1f92fb443782e00dd0cf3dcf2f" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37487 } } }
I20260812 03:52:45.783555 23761 sys_catalog.cc:458] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:45.783959 23771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:45.784504 23771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:45.787127 23771 catalog_manager.cc:1347] Generated new cluster ID: 72cc73cecff744eda7b70f721a323637
I20260812 03:52:45.787202 23771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:45.813222 23771 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:45.813823 23771 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:45.818698 23771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b457fc1f92fb443782e00dd0cf3dcf2f: Generated new TSK 0
I20260812 03:52:45.818910 23771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:45.832502 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:45.828014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51244 (local address 127.17.54.254:37487)
0812 03:52:45.828086 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:45.828088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:45.828181 (+ 93us) server_negotiation.cc:407] Connection header received
0812 03:52:45.828269 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:45.828271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:45.828284 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:45.828375 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:52:45.828767 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.829611 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.830560 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.830879 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.830955 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.831196 (+ 241us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:52:45.832248 (+ 1052us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:52:45.832402 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51244: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 03:52:45.832602 23788 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:45.827914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37487 (local address 127.0.0.1:51244)
0812 03:52:45.828106 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:45.828117 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:45.828183 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:45.828488 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:45.828491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:45.828497 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:52:45.828629 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:45.828633 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.829755 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.829757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:45.830475 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:45.830478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:45.831007 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:45.831009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:45.831069 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:45.831076 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:52:45.832368 (+ 1292us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:52:45.832479 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.54.254:37487: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":122,"threads_started":1}
I20260812 03:52:45.832918 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 23676
E20260812 03:52:45.845969 23789 webserver.cc:533] Webserver: sslize failed:
E20260812 03:52:45.846071 23789 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T03:52:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T03:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:45Z Disabled control of system clock
I20260812 03:52:45.888526 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:41479
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:41479
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/testchainca.cert with env {}
W20260812 03:52:45.977859 23793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:45.978096 23793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:45.978120 23793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:45.979622 23793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:45.979686 23793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:45.979699 23793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:45.979712 23793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:45.979725 23793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:52:45.979737 23793 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:52:45.979786 23793 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 03:52:45.981472 23793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:41479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:41479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:45.981719 23793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:45.982115 23793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:45.987836 23799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:45.987856 23802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:45.987913 23793 server_base.cc:1034] running on GCE node
W20260812 03:52:45.987900 23800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:45.988375 23793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:45.988715 23793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:45.989886 23793 hybrid_clock.cc:648] HybridClock initialized: now 1786506765989863 us; error 41 us; skew 500 ppm
I20260812 03:52:45.991372 23793 webserver.cc:460] Webserver started at http://127.17.54.254:37365/ using document root <none> and password file <none>
I20260812 03:52:45.991652 23793 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:45.991703 23793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:45.991873 23793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:45.993008 23793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "32e543550f1143a085297d02c5366648"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "b0bf74030b81f9e33151ae1388469752"
server_key_iv: "47d3c57772e7f48798a7985924fb0432"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.993381 23793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "32e543550f1143a085297d02c5366648"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "b0bf74030b81f9e33151ae1388469752"
server_key_iv: "47d3c57772e7f48798a7985924fb0432"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.994906 23793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:45.995987 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:45.996194 23793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:45.996313 23793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "32e543550f1143a085297d02c5366648"
format_stamp: "Formatted at 2026-08-12 03:52:45 on dist-test-slave-tm1g"
server_key: "b0bf74030b81f9e33151ae1388469752"
server_key_iv: "47d3c57772e7f48798a7985924fb0432"
server_key_version: "encryptionkey@0"
I20260812 03:52:45.996394 23793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:46.007028 23793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:46.007449 23793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:46.007751 23793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:46.012626 23793 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:41479
I20260812 03:52:46.012739 23868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:41479 every 8 connection(s)
I20260812 03:52:46.013101 23793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:46.014014 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 23793
I20260812 03:52:46.014159 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:46.014393 23869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:46.014410 17627 external_mini_cluster.cc:1428] Setting key 9a955e2921abd3c91b7b8439a26cbd78
I20260812 03:52:46.018098 23869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648: Bootstrap starting.
I20260812 03:52:46.018919 23869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:46.019268 23869 log.cc:824] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:46.020152 23869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648: No bootstrap required, opened a new log
I20260812 03:52:46.021230 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:46.014987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.17.54.254:41479)
0812 03:52:46.015186 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:46.015194 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:46.015219 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:52:46.015620 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:46.015627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:46.015690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:46.015805 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:46.016112 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:46.017051 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:46.018171 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:46.018717 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:46.018813 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:46.020384 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:46.020390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:46.020390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:46.020632 (+ 242us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:46.020707 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:46.020726 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:46.020799 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:46.020900 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:52:46.020990 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260812 03:52:46.022580 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:46.022822 23869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e543550f1143a085297d02c5366648" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41479 } }
I20260812 03:52:46.023001 23869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:46.023051 23869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e543550f1143a085297d02c5366648, State: Initialized, Role: FOLLOWER
I20260812 03:52:46.023195 23869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e543550f1143a085297d02c5366648" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41479 } }
I20260812 03:52:46.023272 23869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:46.023311 23869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:46.023370 23869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:46.077126 23869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e543550f1143a085297d02c5366648" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41479 } }
I20260812 03:52:46.077369 23869 leader_election.cc:302] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32e543550f1143a085297d02c5366648; no voters:
I20260812 03:52:46.077632 23869 leader_election.cc:288] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:46.077813 23874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:46.078032 23869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:46.078059 23874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [term 1 LEADER]: Becoming Leader. State: Replica: 32e543550f1143a085297d02c5366648, State: Running, Role: LEADER
I20260812 03:52:46.078293 23874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e543550f1143a085297d02c5366648" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41479 } }
I20260812 03:52:46.078734 23875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32e543550f1143a085297d02c5366648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e543550f1143a085297d02c5366648" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41479 } } }
I20260812 03:52:46.078763 23877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32e543550f1143a085297d02c5366648. Latest consensus state: current_term: 1 leader_uuid: "32e543550f1143a085297d02c5366648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e543550f1143a085297d02c5366648" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41479 } } }
I20260812 03:52:46.078969 23877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:46.078887 23875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:46.079355 23886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:46.080118 23886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:46.083425 23886 catalog_manager.cc:1347] Generated new cluster ID: 54a02ccfebd54bf5b5df654462c431c5
I20260812 03:52:46.083518 23886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:46.102237 23886 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:46.102982 23886 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:46.107029 23886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32e543550f1143a085297d02c5366648: Generated new TSK 0
I20260812 03:52:46.107281 23886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:46.177999 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:46.116730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.17.54.254:41479)
0812 03:52:46.116798 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:46.116801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:46.116915 (+ 114us) server_negotiation.cc:407] Connection header received
0812 03:52:46.116982 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:46.116983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:46.116998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:46.117095 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:52:46.117231 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:46.118069 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:46.119186 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:46.119513 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:46.119591 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:46.119906 (+ 315us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:52:46.177559 (+ 57653us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:52:46.177861 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44280: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 03:52:46.178115 23903 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:46.116626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41479 (local address 127.0.0.1:44280)
0812 03:52:46.116849 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:46.116859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:46.116915 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:46.117104 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:46.117105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:46.117107 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:52:46.117164 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:46.117166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:46.118209 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:46.118210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:46.119091 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:46.119094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:46.119632 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:46.119633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:46.119701 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:46.119708 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:52:46.177803 (+ 58095us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:52:46.177967 (+ 164us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.54.254:41479: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":140,"threads_started":1}
W20260812 03:52:46.178323 23872 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44280: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260812 03:52:46.178794 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 23793
2026-08-12T03:52:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (328 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (106 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (104 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:47 dist-test-slave-tm1g krb5kdc[23919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:47 dist-test-slave-tm1g krb5kdc[23919](info): set up 2 sockets
Aug 12 03:52:47 dist-test-slave-tm1g krb5kdc[23919](info): commencing operation
krb5kdc: starting...
W20260812 03:52:48.489239 17627 mini_kdc.cc:121] Time spent starting KDC: real 2.073s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:48 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:48.625998 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:41969
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:41969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:48.716197 23934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:48.716701 23934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:48.716792 23934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:48.718544 23934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:48.718606 23934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:48.718633 23934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:48.718659 23934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:48.718729 23934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:48.721299 23934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:41969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:48.721652 23934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:48.721940 23934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:48.727766 23941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:48.727758 23943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:48.728191 23934 server_base.cc:1034] running on GCE node
W20260812 03:52:48.727777 23940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:48.728533 23934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:48.729157 23934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:48.730386 23934 hybrid_clock.cc:648] HybridClock initialized: now 1786506768730364 us; error 35 us; skew 500 ppm
Aug 12 03:52:48 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:48.733620 23934 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:48.734061 23934 webserver.cc:460] Webserver started at http://127.17.54.254:35675/ using document root <none> and password file <none>
I20260812 03:52:48.734274 23934 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:48.734373 23934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:48.734519 23934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:48.735939 23934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "635c607a795f43ed8dbdf6046612dace"
format_stamp: "Formatted at 2026-08-12 03:52:48 on dist-test-slave-tm1g"
server_key: "74f9e3a98c3b7695fd6d5a2ac76dde13"
server_key_iv: "026bdc8215afa32692e890e1e5a46046"
server_key_version: "encryptionkey@0"
I20260812 03:52:48.736409 23934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "635c607a795f43ed8dbdf6046612dace"
format_stamp: "Formatted at 2026-08-12 03:52:48 on dist-test-slave-tm1g"
server_key: "74f9e3a98c3b7695fd6d5a2ac76dde13"
server_key_iv: "026bdc8215afa32692e890e1e5a46046"
server_key_version: "encryptionkey@0"
I20260812 03:52:48.738443 23934 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 03:52:48.739693 23950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:48.739904 23934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:48.740010 23934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "635c607a795f43ed8dbdf6046612dace"
format_stamp: "Formatted at 2026-08-12 03:52:48 on dist-test-slave-tm1g"
server_key: "74f9e3a98c3b7695fd6d5a2ac76dde13"
server_key_iv: "026bdc8215afa32692e890e1e5a46046"
server_key_version: "encryptionkey@0"
I20260812 03:52:48.740098 23934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:48.753988 23934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:48.757007 23934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:48.757185 23934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:48.762265 23934 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:41969
I20260812 03:52:48.762346 24011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:41969 every 8 connection(s)
I20260812 03:52:48.762735 23934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:48.763732 24012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:48.766204 24012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace: Bootstrap starting.
I20260812 03:52:48.766892 24012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:48.767228 24012 log.cc:824] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:48.768006 24012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace: No bootstrap required, opened a new log
I20260812 03:52:48.769759 24012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c607a795f43ed8dbdf6046612dace" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41969 } }
I20260812 03:52:48.769908 24012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:48.769932 24012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635c607a795f43ed8dbdf6046612dace, State: Initialized, Role: FOLLOWER
I20260812 03:52:48.770124 24012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c607a795f43ed8dbdf6046612dace" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41969 } }
I20260812 03:52:48.770195 24012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:48.770217 24012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:48.770282 24012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:48.770958 24012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c607a795f43ed8dbdf6046612dace" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41969 } }
I20260812 03:52:48.771050 24012 leader_election.cc:302] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 635c607a795f43ed8dbdf6046612dace; no voters:
I20260812 03:52:48.771270 24012 leader_election.cc:288] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:48.771322 24016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:48.771564 24012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:48.771833 24016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [term 1 LEADER]: Becoming Leader. State: Replica: 635c607a795f43ed8dbdf6046612dace, State: Running, Role: LEADER
I20260812 03:52:48.771939 24016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c607a795f43ed8dbdf6046612dace" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41969 } }
I20260812 03:52:48.771991 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 23934
I20260812 03:52:48.772083 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:48.772277 17627 external_mini_cluster.cc:1428] Setting key 5ed3c983a6115cbfd7477000ed47f439
I20260812 03:52:48.772351 24017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "635c607a795f43ed8dbdf6046612dace" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c607a795f43ed8dbdf6046612dace" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41969 } } }
I20260812 03:52:48.772373 24018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635c607a795f43ed8dbdf6046612dace. Latest consensus state: current_term: 1 leader_uuid: "635c607a795f43ed8dbdf6046612dace" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c607a795f43ed8dbdf6046612dace" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41969 } } }
I20260812 03:52:48.772445 24017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:48.772466 24018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:48.772691 24028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:48.773236 24028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:48.775132 24028 catalog_manager.cc:1347] Generated new cluster ID: b6f117f91da14152afb32f0213f97a10
I20260812 03:52:48.775192 24028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:52:48 dist-test-slave-tm1g krb5kdc[23919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:48.783330 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:48.772923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.17.54.254:41969)
0812 03:52:48.773498 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:48.773508 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:48.773545 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:52:48.773676 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:48.773684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:48.773745 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:48.773892 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:48.774721 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:48.775803 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:48.776574 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:48.777022 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:48.777152 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:48.779896 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:48.779901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:48.779901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:48.779919 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:48.781696 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:48.782083 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:48.782087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:48.782087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:48.782187 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:48.782409 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:48.782410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:48.782410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:48.782680 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:48.782857 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:48.782939 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:52:48.783120 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":76,"threads_started":1}
I20260812 03:52:48.793804 24028 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:48.794607 24028 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:48.806861 24028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 635c607a795f43ed8dbdf6046612dace: Generated new TSK 0
I20260812 03:52:48.807107 24028 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:48.843330 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41969
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:48.929921 24043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:48.930135 24043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:48.930160 24043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:48.931408 24043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:48.931457 24043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:48.931486 24043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:48.933043 24043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:48.933305 24043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:48.933554 24043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:48.938994 24050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:48.938962 24051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:48.939145 24043 server_base.cc:1034] running on GCE node
W20260812 03:52:48.938970 24053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:48.939455 24043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:48.939751 24043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:48.940920 24043 hybrid_clock.cc:648] HybridClock initialized: now 1786506768940891 us; error 53 us; skew 500 ppm
Aug 12 03:52:48 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:48.943846 24043 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:48.944361 24043 webserver.cc:460] Webserver started at http://127.17.54.193:46331/ using document root <none> and password file <none>
I20260812 03:52:48.944718 24043 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:48.944799 24043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:48.944949 24043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:48.946189 24043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "87f970fca15543a0bdf7f9e4649f30b2"
format_stamp: "Formatted at 2026-08-12 03:52:48 on dist-test-slave-tm1g"
server_key: "15feddb7aa373489fac5e67627f485c8"
server_key_iv: "e222a59a1e35724ba19484b5fae27b55"
server_key_version: "encryptionkey@0"
I20260812 03:52:48.946607 24043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "87f970fca15543a0bdf7f9e4649f30b2"
format_stamp: "Formatted at 2026-08-12 03:52:48 on dist-test-slave-tm1g"
server_key: "15feddb7aa373489fac5e67627f485c8"
server_key_iv: "e222a59a1e35724ba19484b5fae27b55"
server_key_version: "encryptionkey@0"
I20260812 03:52:48.948154 24043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:48.949167 24060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:48.949368 24043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:48.949476 24043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "87f970fca15543a0bdf7f9e4649f30b2"
format_stamp: "Formatted at 2026-08-12 03:52:48 on dist-test-slave-tm1g"
server_key: "15feddb7aa373489fac5e67627f485c8"
server_key_iv: "e222a59a1e35724ba19484b5fae27b55"
server_key_version: "encryptionkey@0"
I20260812 03:52:48.949564 24043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:48.967741 24043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:48.969751 24043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:48.969929 24043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:48.970178 24043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:48.970527 24043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:48.970582 24043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:48.970625 24043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:48.970656 24043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:48.976949 24043 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:33283
I20260812 03:52:48.976984 24187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:33283 every 8 connection(s)
I20260812 03:52:48.977327 24043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:48.979578 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 24043
I20260812 03:52:48.979691 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:48.979928 17627 external_mini_cluster.cc:1428] Setting key 3fd4f79d801d1ea3d0efcc5c0ddeafe2
Aug 12 03:52:48 dist-test-slave-tm1g krb5kdc[23919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:48.988138 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:48.978052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:35063 (local address 127.17.54.254:41969)
0812 03:52:48.978104 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:48.978106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:48.978284 (+ 178us) server_negotiation.cc:407] Connection header received
0812 03:52:48.979277 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:48.979278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:48.979291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:48.979363 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:48.980301 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:48.980919 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:48.982526 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:48.982892 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:48.982968 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:48.985194 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:48.985195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:48.985195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:48.985208 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:48.986703 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:48.987147 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:48.987148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:48.987148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:48.987207 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:48.987461 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:48.987462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:48.987463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:48.987584 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:48.987653 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:48.987950 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 03:52:48.988044 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":23}
I20260812 03:52:48.988325 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:48.977994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41969 (local address 127.17.54.193:35063)
0812 03:52:48.978163 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:48.978208 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:48.979180 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:48.979552 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:48.979557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:48.979943 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:48.980210 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:48.980215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:48.981010 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:48.981011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:48.982422 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:48.982426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:48.982996 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:48.982997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:48.983113 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:48.983701 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:48.983709 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:48.985088 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:48.986814 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:48.986821 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:48.986824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:48.987055 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:48.987273 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:48.987274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:48.987275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:48.987388 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:48.987656 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:48.987664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:48.987812 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 03:52:48.987913 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:52:48.988083 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
I20260812 03:52:48.989063 24188 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41969
I20260812 03:52:48.989154 24188 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:48.989303 24188 heartbeater.cc:511] Master 127.17.54.254:41969 requested a full tablet report, sending...
I20260812 03:52:48.989785 23967 ts_manager.cc:194] Registered new tserver with Master: 87f970fca15543a0bdf7f9e4649f30b2 (127.17.54.193:33283)
I20260812 03:52:48.990541 23967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:35063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:49.009836 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41969
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:49.094805 24195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:49.095065 24195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:49.095106 24195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:49.096313 24195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:49.096386 24195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:49.096436 24195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:49.098053 24195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:49.098397 24195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:49.098657 24195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:49.104359 24203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:49.104359 24200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:49.104374 24201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:49.104719 24195 server_base.cc:1034] running on GCE node
I20260812 03:52:49.105010 24195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:49.105372 24195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:49.106563 24195 hybrid_clock.cc:648] HybridClock initialized: now 1786506769106547 us; error 44 us; skew 500 ppm
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.109226 24195 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:52:49.109705 24195 webserver.cc:460] Webserver started at http://127.17.54.194:46077/ using document root <none> and password file <none>
I20260812 03:52:49.109929 24195 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:49.109975 24195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:49.110069 24195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:49.111173 24195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "09503ea33a474b4aab981ae32cb081ad"
format_stamp: "Formatted at 2026-08-12 03:52:49 on dist-test-slave-tm1g"
server_key: "d751e670f0394e09ba5bf3a54a5dcbbb"
server_key_iv: "90617c2277010cc43fb50bf4ad386ee9"
server_key_version: "encryptionkey@0"
I20260812 03:52:49.111490 24195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "09503ea33a474b4aab981ae32cb081ad"
format_stamp: "Formatted at 2026-08-12 03:52:49 on dist-test-slave-tm1g"
server_key: "d751e670f0394e09ba5bf3a54a5dcbbb"
server_key_iv: "90617c2277010cc43fb50bf4ad386ee9"
server_key_version: "encryptionkey@0"
I20260812 03:52:49.112843 24195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:52:49.113626 24211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:49.113847 24195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:49.113955 24195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "09503ea33a474b4aab981ae32cb081ad"
format_stamp: "Formatted at 2026-08-12 03:52:49 on dist-test-slave-tm1g"
server_key: "d751e670f0394e09ba5bf3a54a5dcbbb"
server_key_iv: "90617c2277010cc43fb50bf4ad386ee9"
server_key_version: "encryptionkey@0"
I20260812 03:52:49.114038 24195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:49.127887 24195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:49.130431 24195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:49.130594 24195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:49.130872 24195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:49.131217 24195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:49.131269 24195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:49.131311 24195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:49.131341 24195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:49.137846 24195 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:44599
I20260812 03:52:49.137890 24340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:44599 every 8 connection(s)
I20260812 03:52:49.138237 24195 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:49.145900 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 24195
I20260812 03:52:49.146030 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:49.146238 17627 external_mini_cluster.cc:1428] Setting key fd7bcc5ada1364239071d98f6077e191
I20260812 03:52:49.149163 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:49.139250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:51019 (local address 127.17.54.254:41969)
0812 03:52:49.139318 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:49.139321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:49.139590 (+ 269us) server_negotiation.cc:407] Connection header received
0812 03:52:49.140536 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:49.140537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:49.140548 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:49.140612 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:49.141431 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.141948 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.143236 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.143441 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.143506 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:49.145884 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:49.145885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:49.145885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:49.145893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:49.147557 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.148003 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.148004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.148005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.148072 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.148362 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.148363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.148363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.148531 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:49.148616 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:49.148971 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 03:52:49.149072 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:52:49.149367 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:49.139220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41969 (local address 127.17.54.194:51019)
0812 03:52:49.139411 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:49.139499 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:49.140425 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:49.140664 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:49.140667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:49.140929 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:49.141151 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:49.141155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.142045 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.142046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:49.143131 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:49.143134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.143520 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.143521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:49.143632 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:49.144217 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:49.144227 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:49.145763 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:49.147667 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:49.147675 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:49.147678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:49.147915 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:49.148136 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:49.148137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:49.148137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:49.148271 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:49.148612 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:49.148618 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:49.148806 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 03:52:49.148944 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 03:52:49.149113 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260812 03:52:49.149802 24341 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41969
I20260812 03:52:49.149916 24341 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:49.150113 24341 heartbeater.cc:511] Master 127.17.54.254:41969 requested a full tablet report, sending...
I20260812 03:52:49.150616 23967 ts_manager.cc:194] Registered new tserver with Master: 09503ea33a474b4aab981ae32cb081ad (127.17.54.194:44599)
I20260812 03:52:49.151036 23967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:51019
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:52:49.177686 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41969
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:49.261783 24346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:49.261993 24346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:49.262017 24346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:49.263212 24346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:49.263257 24346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:49.263284 24346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:49.264748 24346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:49.264959 24346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:49.265147 24346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:49.270382 24352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:49.270422 24355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:49.270506 24346 server_base.cc:1034] running on GCE node
W20260812 03:52:49.270388 24353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:49.270818 24346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:49.271106 24346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:49.272315 24346 hybrid_clock.cc:648] HybridClock initialized: now 1786506769272302 us; error 32 us; skew 500 ppm
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.275050 24346 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:52:49.275556 24346 webserver.cc:460] Webserver started at http://127.17.54.195:42013/ using document root <none> and password file <none>
I20260812 03:52:49.275811 24346 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:49.275884 24346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:49.276036 24346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:49.277094 24346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "3090dc3179134385953286948722a833"
format_stamp: "Formatted at 2026-08-12 03:52:49 on dist-test-slave-tm1g"
server_key: "c9bb3340945068fd8e209d9f1bfcb40a"
server_key_iv: "787ab5f2cd35ff652c6145ded4a8cacd"
server_key_version: "encryptionkey@0"
I20260812 03:52:49.277463 24346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "3090dc3179134385953286948722a833"
format_stamp: "Formatted at 2026-08-12 03:52:49 on dist-test-slave-tm1g"
server_key: "c9bb3340945068fd8e209d9f1bfcb40a"
server_key_iv: "787ab5f2cd35ff652c6145ded4a8cacd"
server_key_version: "encryptionkey@0"
I20260812 03:52:49.278899 24346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:49.279652 24366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:49.279873 24346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:49.279979 24346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "3090dc3179134385953286948722a833"
format_stamp: "Formatted at 2026-08-12 03:52:49 on dist-test-slave-tm1g"
server_key: "c9bb3340945068fd8e209d9f1bfcb40a"
server_key_iv: "787ab5f2cd35ff652c6145ded4a8cacd"
server_key_version: "encryptionkey@0"
I20260812 03:52:49.280052 24346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:49.287106 24346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:49.289057 24346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:49.289214 24346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:49.289489 24346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:49.289811 24346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:49.289863 24346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:49.289906 24346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:49.289928 24346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:49.296334 24346 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:45759
I20260812 03:52:49.296381 24493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:45759 every 8 connection(s)
I20260812 03:52:49.296720 24346 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:49.303498 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 24346
I20260812 03:52:49.303601 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:49.303807 17627 external_mini_cluster.cc:1428] Setting key e391196abe7a42d7a40ab7b531d69e20
I20260812 03:52:49.306759 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:49.297431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:49925 (local address 127.17.54.254:41969)
0812 03:52:49.297505 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:49.297507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:49.297725 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:52:49.298775 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:49.298776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:49.298787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:49.298853 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:49.299528 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.300030 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.301136 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.301323 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.301376 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:49.303617 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:49.303619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:49.303623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:49.303644 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:49.305243 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.305682 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.305683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.305684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.305750 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.306001 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.306002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.306002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.306132 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:49.306207 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:49.306573 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 03:52:49.306665 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:52:49.307004 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:49.297399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41969 (local address 127.17.54.195:49925)
0812 03:52:49.297591 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:49.297635 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:49.298678 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:49.298947 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:49.298950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:49.299225 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:49.299446 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:49.299450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.300107 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.300108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:49.301055 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:49.301058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.301394 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.301395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:49.301518 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:49.302041 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:49.302049 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:49.303509 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:49.305349 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:49.305356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:49.305359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:49.305604 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:49.305815 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:49.305816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:49.305817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:49.305936 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:49.306198 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:49.306205 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:49.306429 (+ 224us) client_negotiation.cc:769] Sending connection context
0812 03:52:49.306545 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 03:52:49.306720 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":112,"threads_started":1}
I20260812 03:52:49.307498 24494 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41969
I20260812 03:52:49.307626 24494 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:49.307807 24494 heartbeater.cc:511] Master 127.17.54.254:41969 requested a full tablet report, sending...
I20260812 03:52:49.308197 23967 ts_manager.cc:194] Registered new tserver with Master: 3090dc3179134385953286948722a833 (127.17.54.195:45759)
I20260812 03:52:49.308593 23967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:49925
I20260812 03:52:49.315979 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/test-admin.keytab.
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.335696 17627 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 03:52:49.337581 24502 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:49.336625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.17.54.193:33283)
0812 03:52:49.336862 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:49.336866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:49.336878 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:49.336906 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:49.336908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:49.336922 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:49.336995 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:49.337492 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43398: BlockingRecv error: recv got EOF from 127.0.0.1:43398 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":106,"threads_started":1}
W20260812 03:52:49.337659 17627 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.338789 17627 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.339743 17627 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:49.348353 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:49.340338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43402 (local address 127.17.54.193:33283)
0812 03:52:49.340404 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:49.340406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:49.340418 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:49.340475 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:49.340476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:49.340486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:49.340534 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:49.340957 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.341700 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.342581 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.342957 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.343024 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:49.345401 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:49.345412 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:49.345413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:49.345422 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:49.347146 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.347492 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.347495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.347495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.347554 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.347750 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.347751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.347751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.347937 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:49.348070 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:49.348106 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 03:52:49.348178 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 03:52:49.350353 24502 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:49.349715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.17.54.193:33283)
0812 03:52:49.349753 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:49.349755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:49.349868 (+ 113us) server_negotiation.cc:407] Connection header received
0812 03:52:49.349916 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:49.349916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:49.349926 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:49.349975 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:49.350253 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43404: BlockingRecv error: recv got EOF from 127.0.0.1:43404 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 03:52:49.350618 17627 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.351634 17627 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.352499 17627 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:49.359593 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:49.352990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.17.54.193:33283)
0812 03:52:49.353054 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:49.353056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:49.353070 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:52:49.353146 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:49.353146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:49.353155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:49.353206 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:49.353538 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.354071 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.354834 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.355064 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.355121 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:49.357185 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:49.357187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:49.357187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:49.357194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:49.358650 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.358964 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.358965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.358965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.359014 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.359156 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.359157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.359157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.359238 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:49.359306 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:49.359391 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 03:52:49.359455 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 03:52:49.361248 24502 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:49.360681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.17.54.193:33283)
0812 03:52:49.360741 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:49.360743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:49.360775 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:52:49.360857 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:49.360858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:49.360868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:49.360943 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:49.361198 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43420: BlockingRecv error: recv got EOF from 127.0.0.1:43420 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 03:52:49.361573 17627 init.cc:244] Error while opening credential cache '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506741478267-17627-0/krb5kdc/krb5cc)
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.362607 17627 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:49.363461 17627 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[23919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:52:49.370402 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:49.363971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43424 (local address 127.17.54.193:33283)
0812 03:52:49.364029 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:49.364030 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:49.364042 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:49.364092 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:49.364093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:49.364100 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:49.364150 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:49.364494 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.364967 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.365655 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:49.365859 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:49.365917 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:49.367996 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:49.367998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:49.367998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:49.368005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:49.369360 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.369658 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.369659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.369659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.369711 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:49.369901 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:49.369902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:49.369902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:49.370017 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:49.370099 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:49.370179 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:52:49.370236 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:52:49.371032 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 24043
I20260812 03:52:49.383456 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 24195
I20260812 03:52:49.390991 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 24346
I20260812 03:52:49.398279 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 23934
2026-08-12T03:52:49Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3005 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[24518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[24518](info): set up 2 sockets
Aug 12 03:52:49 dist-test-slave-tm1g krb5kdc[24518](info): commencing operation
krb5kdc: starting...
W20260812 03:52:50.534732 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:50 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.254.keytab.
Entry for principal oryx/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.254.keytab.
I20260812 03:52:50.669907 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42117
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.254.keytab
--principal=oryx/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:42117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:50.755728 24531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:50.755970 24531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:50.755996 24531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:50.757220 24531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:50.757262 24531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:50.757277 24531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:50.757289 24531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:50.757302 24531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:50.759255 24531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:42117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.254.keytab
--principal=oryx/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:50.759560 24531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:50.759768 24531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:50.765205 24538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:50.765230 24540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:50.765278 24537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:50.765517 24531 server_base.cc:1034] running on GCE node
I20260812 03:52:50.765771 24531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:50.766045 24531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:50.767210 24531 hybrid_clock.cc:648] HybridClock initialized: now 1786506770767183 us; error 44 us; skew 500 ppm
Aug 12 03:52:50 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:50.769969 24531 init.cc:377] Logged in from keytab as oryx/127.17.54.254@KRBTEST.COM (short username oryx)
I20260812 03:52:50.770574 24531 webserver.cc:460] Webserver started at http://127.17.54.254:45217/ using document root <none> and password file <none>
I20260812 03:52:50.770840 24531 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:50.770936 24531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:50.771097 24531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:50.772207 24531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "25d39d64d04b4b71b2e72682026b02a1"
format_stamp: "Formatted at 2026-08-12 03:52:50 on dist-test-slave-tm1g"
server_key: "af5ebb7342586bf14b08975f9a2c305f"
server_key_iv: "ab9e728e23c455b60e55a62848a15549"
server_key_version: "encryptionkey@0"
I20260812 03:52:50.772588 24531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "25d39d64d04b4b71b2e72682026b02a1"
format_stamp: "Formatted at 2026-08-12 03:52:50 on dist-test-slave-tm1g"
server_key: "af5ebb7342586bf14b08975f9a2c305f"
server_key_iv: "ab9e728e23c455b60e55a62848a15549"
server_key_version: "encryptionkey@0"
I20260812 03:52:50.773995 24531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:50.774896 24547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:50.775151 24531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:50.775250 24531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "25d39d64d04b4b71b2e72682026b02a1"
format_stamp: "Formatted at 2026-08-12 03:52:50 on dist-test-slave-tm1g"
server_key: "af5ebb7342586bf14b08975f9a2c305f"
server_key_iv: "ab9e728e23c455b60e55a62848a15549"
server_key_version: "encryptionkey@0"
I20260812 03:52:50.775347 24531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:50.792152 24531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:50.794204 24531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:50.794410 24531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:50.798804 24531 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:42117
I20260812 03:52:50.798843 24608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:42117 every 8 connection(s)
I20260812 03:52:50.799225 24531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:50.800287 24609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:50.802768 24609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: Bootstrap starting.
I20260812 03:52:50.803395 24609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:50.803717 24609 log.cc:824] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:50.804550 24609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: No bootstrap required, opened a new log
I20260812 03:52:50.806151 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 24531
I20260812 03:52:50.806283 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:50.806535 17627 external_mini_cluster.cc:1428] Setting key 85749159687241db6122bd75b0061a75
I20260812 03:52:50.806577 24609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } }
I20260812 03:52:50.806761 24609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:50.806833 24609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d39d64d04b4b71b2e72682026b02a1, State: Initialized, Role: FOLLOWER
I20260812 03:52:50.807024 24609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } }
I20260812 03:52:50.807147 24609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:50.807206 24609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:50.807274 24609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:50.808583 24609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } }
I20260812 03:52:50.808710 24609 leader_election.cc:302] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25d39d64d04b4b71b2e72682026b02a1; no voters:
I20260812 03:52:50.808898 24609 leader_election.cc:288] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:50.808954 24616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:50.809147 24616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 1 LEADER]: Becoming Leader. State: Replica: 25d39d64d04b4b71b2e72682026b02a1, State: Running, Role: LEADER
I20260812 03:52:50.809198 24609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:50.810019 24616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } }
I20260812 03:52:50.810797 24618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } }
I20260812 03:52:50.810916 24617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25d39d64d04b4b71b2e72682026b02a1. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } }
I20260812 03:52:50.810989 24617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:50.810983 24618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
W20260812 03:52:50.811450 24632 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:50.811578 24632 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:50.811961 24633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:50.812165 24633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:50.813934 24633 catalog_manager.cc:1347] Generated new cluster ID: 7b6f610943c94719ac39e64ebffbf72d
I20260812 03:52:50.813992 24633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:52:50 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:50.819378 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:50.807169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.17.54.254:42117)
0812 03:52:50.807390 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:50.807399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:50.807472 (+ 73us) server_negotiation.cc:407] Connection header received
0812 03:52:50.807620 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:50.807625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:50.807682 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:50.807813 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:50.808734 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:50.810350 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:50.811495 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:50.812155 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:50.812282 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:50.815346 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:50.815351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:50.815351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:50.815364 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:50.817465 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:50.817995 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:50.817998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:50.817998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:50.818138 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:50.818386 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:50.818387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:50.818387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:50.818718 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:50.818936 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:50.819007 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 03:52:50.819157 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260812 03:52:50.821519 24633 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:50.822266 24633 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:50.827929 24633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: Generated new TSK 0
I20260812 03:52:50.828152 24633 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab.
Entry for principal oryx/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab.
I20260812 03:52:50.857981 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42117
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab
--principal=oryx/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:50.944976 24640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:50.945300 24640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:50.945348 24640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:50.947499 24640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:50.947574 24640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:50.947623 24640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:50.950167 24640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab
--principal=oryx/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:50.950853 24640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:50.951551 24640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:50.957527 24649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:50.957537 24647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:50.957564 24640 server_base.cc:1034] running on GCE node
W20260812 03:52:50.957537 24646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:50.958020 24640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:50.958356 24640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:50.959558 24640 hybrid_clock.cc:648] HybridClock initialized: now 1786506770959541 us; error 26 us; skew 500 ppm
Aug 12 03:52:50 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:50.962445 24640 init.cc:377] Logged in from keytab as oryx/127.17.54.193@KRBTEST.COM (short username oryx)
I20260812 03:52:50.962949 24640 webserver.cc:460] Webserver started at http://127.17.54.193:39737/ using document root <none> and password file <none>
I20260812 03:52:50.963217 24640 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:50.963302 24640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:50.963461 24640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:50.964661 24640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "b2c3658deb024df3891ff955776281c4"
format_stamp: "Formatted at 2026-08-12 03:52:50 on dist-test-slave-tm1g"
server_key: "c29e5ed3714b55d2824a722820644b46"
server_key_iv: "802491f5ae8717dd631ffe497c053fc3"
server_key_version: "encryptionkey@0"
I20260812 03:52:50.965076 24640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "b2c3658deb024df3891ff955776281c4"
format_stamp: "Formatted at 2026-08-12 03:52:50 on dist-test-slave-tm1g"
server_key: "c29e5ed3714b55d2824a722820644b46"
server_key_iv: "802491f5ae8717dd631ffe497c053fc3"
server_key_version: "encryptionkey@0"
I20260812 03:52:50.966704 24640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:50.967586 24656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:50.967783 24640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:50.967896 24640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "b2c3658deb024df3891ff955776281c4"
format_stamp: "Formatted at 2026-08-12 03:52:50 on dist-test-slave-tm1g"
server_key: "c29e5ed3714b55d2824a722820644b46"
server_key_iv: "802491f5ae8717dd631ffe497c053fc3"
server_key_version: "encryptionkey@0"
I20260812 03:52:50.967999 24640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:50.977861 24640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:50.980324 24640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:50.980511 24640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:50.980770 24640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:50.981117 24640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:50.981177 24640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:50.981225 24640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:50.981266 24640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:50.987610 24640 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:46153
I20260812 03:52:50.987658 24782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:46153 every 8 connection(s)
I20260812 03:52:50.987991 24640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:52:50 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:50.994225 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 24640
I20260812 03:52:50.994354 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:50.994571 17627 external_mini_cluster.cc:1428] Setting key e8b474f95b617ff8a86058020a4e616c
I20260812 03:52:50.997886 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:50.988753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:35445 (local address 127.17.54.254:42117)
0812 03:52:50.988855 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:50.988857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:50.989040 (+ 183us) server_negotiation.cc:407] Connection header received
0812 03:52:50.989528 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:50.989529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:50.989551 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:50.989619 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:50.990625 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:50.991145 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:50.992126 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:50.992343 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:50.992404 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:50.994783 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:50.994784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:50.994785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:50.994792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:50.996239 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:50.996781 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:50.996782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:50.996782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:50.996858 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:50.997177 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:50.997178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:50.997178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:50.997350 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:50.997427 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:50.997686 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 03:52:50.997785 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:52:50.997960 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:50.988715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.193:35445)
0812 03:52:50.988898 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:50.988943 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:50.989440 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:50.989670 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:50.989674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:50.990262 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:50.990539 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:50.990545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:50.991227 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:50.991228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:50.992037 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:50.992039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:50.992423 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:50.992424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:50.992527 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:50.993066 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:50.993074 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:50.994671 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:50.996332 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:50.996340 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:50.996342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:50.996694 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:50.996941 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:50.996942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:50.996942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:50.997101 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:50.997420 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:50.997426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:50.997546 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 03:52:50.997638 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:52:50.997766 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260812 03:52:50.998502 24784 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42117
I20260812 03:52:50.998616 24784 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:50.998801 24784 heartbeater.cc:511] Master 127.17.54.254:42117 requested a full tablet report, sending...
I20260812 03:52:50.999380 24564 ts_manager.cc:194] Registered new tserver with Master: b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153)
I20260812 03:52:51.000396 24564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:35445
WARNING: no policy specified for oryx/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab.
Entry for principal oryx/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab.
I20260812 03:52:51.027266 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42117
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab
--principal=oryx/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:51.113346 24789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:51.113569 24789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:51.113595 24789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:51.114786 24789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:51.114826 24789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:51.114853 24789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:51.116361 24789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab
--principal=oryx/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:51.116569 24789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:51.116760 24789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:51.121996 24795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.122020 24798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.121996 24794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:51.122293 24789 server_base.cc:1034] running on GCE node
I20260812 03:52:51.122607 24789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:51.122874 24789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:51.124053 24789 hybrid_clock.cc:648] HybridClock initialized: now 1786506771124038 us; error 30 us; skew 500 ppm
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:51.126890 24789 init.cc:377] Logged in from keytab as oryx/127.17.54.194@KRBTEST.COM (short username oryx)
I20260812 03:52:51.127364 24789 webserver.cc:460] Webserver started at http://127.17.54.194:44695/ using document root <none> and password file <none>
I20260812 03:52:51.127612 24789 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:51.127683 24789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:51.127832 24789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:51.128966 24789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "9b21f85a09e64588806be1c1e5ce1c79"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "565a7ec4d949f69dc84270f946d7b167"
server_key_iv: "27939f32c19e65717ea3050484f5b0cc"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.129343 24789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "9b21f85a09e64588806be1c1e5ce1c79"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "565a7ec4d949f69dc84270f946d7b167"
server_key_iv: "27939f32c19e65717ea3050484f5b0cc"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.130836 24789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:51.131695 24806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.131877 24789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:51.131997 24789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "9b21f85a09e64588806be1c1e5ce1c79"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "565a7ec4d949f69dc84270f946d7b167"
server_key_iv: "27939f32c19e65717ea3050484f5b0cc"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.132085 24789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:51.144887 24789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:51.147109 24789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:51.147282 24789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:51.147537 24789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:51.147864 24789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:51.147917 24789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.147960 24789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:51.147991 24789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.154086 24789 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:42359
I20260812 03:52:51.154119 24935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:42359 every 8 connection(s)
I20260812 03:52:51.154537 24789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:51.164366 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 24789
I20260812 03:52:51.164481 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:51.164634 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.155366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:44727 (local address 127.17.54.254:42117)
0812 03:52:51.155449 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.155451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.155672 (+ 221us) server_negotiation.cc:407] Connection header received
0812 03:52:51.156178 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.156179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.156190 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.156251 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.157262 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.157841 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.158959 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.159161 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.159223 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.161705 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.161706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.161706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.161713 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.163166 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.163600 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.163601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.163602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.163659 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.163932 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.163933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.163934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.164066 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.164176 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.164414 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.164522 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:52:51.164695 17627 external_mini_cluster.cc:1428] Setting key 7c7054eef363dcb7e2685ad36cfd9b4d
I20260812 03:52:51.164774 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.155266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.194:44727)
0812 03:52:51.155521 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.155560 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.156070 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.156329 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.156332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.156962 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.157161 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.157165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.157939 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.157940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.158877 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.158880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.159257 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.159258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.159374 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.159962 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.159972 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.161621 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.163267 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.163275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.163277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.163530 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.163721 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.163721 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.163722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.163833 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.164161 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.164166 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.164285 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.164370 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.164548 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
I20260812 03:52:51.165316 24936 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42117
I20260812 03:52:51.165449 24936 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:51.165628 24936 heartbeater.cc:511] Master 127.17.54.254:42117 requested a full tablet report, sending...
I20260812 03:52:51.166054 24564 ts_manager.cc:194] Registered new tserver with Master: 9b21f85a09e64588806be1c1e5ce1c79 (127.17.54.194:42359)
I20260812 03:52:51.166594 24564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:44727
WARNING: no policy specified for oryx/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab.
Entry for principal oryx/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab.
I20260812 03:52:51.194149 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42117
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab
--principal=oryx/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:51.279296 24941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:51.279560 24941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:51.279603 24941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:51.280840 24941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:51.280921 24941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:51.280968 24941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:51.282547 24941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab
--principal=oryx/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:51.282852 24941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:51.283107 24941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:51.288451 24946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:51.288677 24941 server_base.cc:1034] running on GCE node
W20260812 03:52:51.288466 24947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.288466 24949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:51.289077 24941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:51.289392 24941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:51.290572 24941 hybrid_clock.cc:648] HybridClock initialized: now 1786506771290560 us; error 27 us; skew 500 ppm
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:51.293228 24941 init.cc:377] Logged in from keytab as oryx/127.17.54.195@KRBTEST.COM (short username oryx)
I20260812 03:52:51.293691 24941 webserver.cc:460] Webserver started at http://127.17.54.195:40837/ using document root <none> and password file <none>
I20260812 03:52:51.293951 24941 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:51.294019 24941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:51.294160 24941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:51.295290 24941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "b801635b38ab4450b56f8e93d80d5a78"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "6546f1809740d448d093a7bf4d143a65"
server_key_iv: "c27588e43a1bdf32084eee533ed0cb3d"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.295699 24941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "b801635b38ab4450b56f8e93d80d5a78"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "6546f1809740d448d093a7bf4d143a65"
server_key_iv: "c27588e43a1bdf32084eee533ed0cb3d"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.297111 24941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:52:51.297910 24956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.298089 24941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:51.298187 24941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "b801635b38ab4450b56f8e93d80d5a78"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "6546f1809740d448d093a7bf4d143a65"
server_key_iv: "c27588e43a1bdf32084eee533ed0cb3d"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.298264 24941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:51.311440 24941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:51.313460 24941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:51.313625 24941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:51.313860 24941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:51.314189 24941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:51.314242 24941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.314284 24941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:51.314445 24941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.320803 24941 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:42841
I20260812 03:52:51.321337 24941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:51.320842 25089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:42841 every 8 connection(s)
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:51.330233 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 24941
I20260812 03:52:51.330372 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:51.330593 17627 external_mini_cluster.cc:1428] Setting key 4f6cdbaabd6afe62fab98d95673e104f
I20260812 03:52:51.332759 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.322378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:38995 (local address 127.17.54.254:42117)
0812 03:52:51.322453 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.322456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.322654 (+ 198us) server_negotiation.cc:407] Connection header received
0812 03:52:51.323173 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.323174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.323186 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.323247 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.324250 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.324797 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.325928 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.326282 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.326398 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.329013 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.329015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.329015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.329022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.330818 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.331331 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.331332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.331333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.331432 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.331954 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.331956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.331956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.332141 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.332205 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.332586 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.332672 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:52:51.332888 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.322348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.195:38995)
0812 03:52:51.322533 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.322572 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.323054 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.323321 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.323325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.323941 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.324146 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.324151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.324886 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.324887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.325822 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.325824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.326408 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.326409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.326610 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.327153 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.327162 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.328767 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.330919 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.330929 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.330931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.331245 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.331590 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.331592 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.331592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.331767 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.332236 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.332244 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.332385 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.332523 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.332671 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260812 03:52:51.333393 25090 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42117
I20260812 03:52:51.333487 25090 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:51.333702 25090 heartbeater.cc:511] Master 127.17.54.254:42117 requested a full tablet report, sending...
I20260812 03:52:51.334146 24564 ts_manager.cc:194] Registered new tserver with Master: b801635b38ab4450b56f8e93d80d5a78 (127.17.54.195:42841)
I20260812 03:52:51.334594 24564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:38995
I20260812 03:52:51.343317 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.54.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.253.keytab.
Entry for principal oryx/127.17.54.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.253.keytab.
I20260812 03:52:51.375041 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.253:44003
--webserver_interface=127.17.54.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.253.keytab
--principal=oryx/127.17.54.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:42117,127.17.54.253:44003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:51.463843 25098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:51.464093 25098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:51.464119 25098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:51.465402 25098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:51.465464 25098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:51.465480 25098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:51.465493 25098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:51.465507 25098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:51.467250 25098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:42117,127.17.54.253:44003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.253.keytab
--principal=oryx/127.17.54.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.253:44003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.54.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
W20260812 03:52:51.467370 25098 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.54.254:42117,127.17.54.253:44003'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 03:52:51.467574 25098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:51.467823 25098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:51.473023 25103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.473011 25108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.473012 25104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:51.473533 25098 server_base.cc:1034] running on GCE node
I20260812 03:52:51.473802 25098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:51.474117 25098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:51.475263 25098 hybrid_clock.cc:648] HybridClock initialized: now 1786506771475228 us; error 54 us; skew 500 ppm
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:51.478040 25098 init.cc:377] Logged in from keytab as oryx/127.17.54.253@KRBTEST.COM (short username oryx)
I20260812 03:52:51.478624 25098 webserver.cc:460] Webserver started at http://127.17.54.253:42401/ using document root <none> and password file <none>
I20260812 03:52:51.478895 25098 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:51.478976 25098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:51.479132 25098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:51.480279 25098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data/instance:
uuid: "43ce326bae49427c8350369649feafb8"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "e1a5b4673ed4c5f3f92aef4b827c171c"
server_key_iv: "fa97c003956e3f6e6532082a7e77e408"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.480665 25098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal/instance:
uuid: "43ce326bae49427c8350369649feafb8"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "e1a5b4673ed4c5f3f92aef4b827c171c"
server_key_iv: "fa97c003956e3f6e6532082a7e77e408"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.482259 25098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:51.483420 25116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.483652 25098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:51.483763 25098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
uuid: "43ce326bae49427c8350369649feafb8"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "e1a5b4673ed4c5f3f92aef4b827c171c"
server_key_iv: "fa97c003956e3f6e6532082a7e77e408"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.483847 25098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:51.495231 25098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:51.497632 25098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:51.497809 25098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:51.502954 25098 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.253:44003
I20260812 03:52:51.502993 25175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.253:44003 every 8 connection(s)
I20260812 03:52:51.503408 25098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
I20260812 03:52:51.504664 25176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:51.506263 25176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:51.511440 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 25098
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.253@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:51.516796 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.507077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.17.54.254:42117)
0812 03:52:51.507148 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.507150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.507368 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:52:51.507887 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.507889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.507903 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.507954 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.508598 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.509171 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.510449 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.510639 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.510699 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.513393 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.513394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.513394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.513402 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.515112 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.515575 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.515576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.515576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.515634 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.515942 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.515944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.515944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.516159 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.516257 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.516572 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.516663 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:52:51.516815 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.507045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:55984)
0812 03:52:51.507246 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.507283 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.507778 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.508029 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.508032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.508285 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.508502 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.508506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.509304 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.509305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.510363 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.510367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.510750 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.510752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.510896 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.511525 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.511548 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.513283 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.515194 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.515203 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.515205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.515501 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.515699 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.515700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.515700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.515849 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.516252 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.516259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.516430 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.516522 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.516647 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":108,"threads_started":1}
I20260812 03:52:51.517577 25176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.54.253@KRBTEST.COM
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.253@KRBTEST.COM for oryx/127.17.54.253@KRBTEST.COM
I20260812 03:52:51.522203 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.511988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.17.54.253:44003)
0812 03:52:51.512159 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.512162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.512212 (+ 50us) server_negotiation.cc:407] Connection header received
0812 03:52:51.512287 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.512290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.512312 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.512397 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.513002 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.513973 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.515194 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.515709 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.515784 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.518768 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.518774 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.518775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.518781 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.520924 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.521305 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.521308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.521308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.521371 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.521595 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.521596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.521596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.521738 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.521931 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.521981 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.522065 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
I20260812 03:52:51.524873 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.517836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.0.0.1:57116)
0812 03:52:51.517889 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.517900 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.517935 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.518267 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.518268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.518487 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.518553 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.518555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.519289 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.519290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.520019 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.520022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.520456 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.520457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.520525 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.520887 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.520890 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.522140 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.523866 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.523866 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.523867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.524042 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.524301 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.524302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.524302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.524360 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.524617 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.524618 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.524657 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.524718 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.524805 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 03:52:51.525028 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.517908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.17.54.253:44003)
0812 03:52:51.518133 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.518134 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.518142 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:51.518159 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.518160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.518169 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.518266 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.518645 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.519194 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.520108 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.520365 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.520445 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.522238 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.522240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.522240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.522247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.523779 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.524156 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.524157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.524157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.524222 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.524427 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.524427 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.524427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.524536 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.524594 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.524743 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.524860 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":202,"thread_start_us":129,"threads_started":1}
I20260812 03:52:51.526587 25176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Bootstrap starting.
I20260812 03:52:51.527207 25176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:51.527469 25176 log.cc:824] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:51.528179 25176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: No bootstrap required, opened a new log
I20260812 03:52:51.529680 25176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } }
I20260812 03:52:51.529804 25176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:51.529829 25176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ce326bae49427c8350369649feafb8, State: Initialized, Role: FOLLOWER
I20260812 03:52:51.529955 25176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } }
I20260812 03:52:51.530222 25186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } } }
I20260812 03:52:51.530328 25186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:51.530361 25176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:52:51.532310 25198 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:51.532378 25198 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:51.534590 25098 master_runner.cc:428] Detected that this master 43ce326bae49427c8350369649feafb8 is joining an existing cluster
I20260812 03:52:51.534713 25098 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.54.253:44003
I20260812 03:52:51.543730 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.536096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55994 (local address 127.17.54.254:42117)
0812 03:52:51.536140 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.536142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.536248 (+ 106us) server_negotiation.cc:407] Connection header received
0812 03:52:51.536332 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.536333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.536345 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.536412 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.536837 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.537658 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.538841 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.539438 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.539515 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.540435 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.540436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.540436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.540443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.542347 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.542798 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.542799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.542800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.542860 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.543111 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.543113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.543113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.543270 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.543318 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.543522 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.543625 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:52:51.543744 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.536076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57130 (local address 127.17.54.253:44003)
0812 03:52:51.536115 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.536117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.536157 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:52:51.536179 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.536180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.536196 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.536258 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.536710 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.537594 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.538542 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.538826 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.539017 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.540495 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.540496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.540497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.540502 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.542613 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.542997 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.542998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.542998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.543047 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.543237 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.543238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.543238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.543351 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.543427 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.543577 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.543653 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:52:51.543740 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.535853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:55994)
0812 03:52:51.536194 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.536202 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.536271 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.536430 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.536431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.536683 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.536749 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.536751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.537742 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.537743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.538742 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.538745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.539541 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.539543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.539617 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.539986 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.539991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.540318 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.542422 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.542423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.542424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.542714 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.542943 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.542944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.542944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.543017 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.543368 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.543369 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.543415 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.543485 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.543599 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":26,"thread_start_us":97,"threads_started":1}
I20260812 03:52:51.544710 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.535853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.0.0.1:57130)
0812 03:52:51.536040 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.536053 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.536110 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.536277 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.536278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.536555 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.536631 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.536632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.537691 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.537692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.538456 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.538458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.539054 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.539056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.539127 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.539831 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.539837 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.540405 (+ 568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.542693 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.542693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.542694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.542937 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.543114 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.543115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.543115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.543170 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.543432 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.543433 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.543473 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.543757 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.543866 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.254@KRBTEST.COM for oryx/127.17.54.253@KRBTEST.COM
I20260812 03:52:51.555675 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.547039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.0.0.1:57140)
0812 03:52:51.547194 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.547243 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.547318 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.547489 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.547492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.547753 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.547808 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.547810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.548727 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.548728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.549383 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.549385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.550272 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.550274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.550423 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.550952 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.550957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.552578 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.554395 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.554402 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.554404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.554611 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.554844 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.554845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.554845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.554914 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.555296 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.555333 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.555388 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.555472 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.555598 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":101,"threads_started":1}
I20260812 03:52:51.555709 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.547083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57140 (local address 127.17.54.253:44003)
0812 03:52:51.547126 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.547128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.547312 (+ 184us) server_negotiation.cc:407] Connection header received
0812 03:52:51.547405 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.547405 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.547417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.547485 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.547876 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.548633 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.549499 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.549804 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.549903 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.552674 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.552676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.552676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.552684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.554269 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.554682 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.554684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.554684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.554750 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.555009 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.555011 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.555011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.555172 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.555268 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.555519 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.555612 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:51.556530 24564 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.54.253:44003: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "43ce326bae49427c8350369649feafb8"
member_type: NON_VOTER
last_known_addr {
host: "127.17.54.253"
port: 44003
}
attrs {
promote: true
}
}
dest_uuid: "25d39d64d04b4b71b2e72682026b02a1"
cas_config_opid_index: -1
I20260812 03:52:51.556926 24564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } }
I20260812 03:52:51.557438 25209 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 43ce326bae49427c8350369649feafb8 (127.17.54.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } }
I20260812 03:52:51.557914 25149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:51.558482 25209 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.54.253:44003
I20260812 03:52:51.558578 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } }
I20260812 03:52:51.558650 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:51.558838 25098 master.cc:537] Master@127.17.54.253:44003 shutting down...
I20260812 03:52:51.558997 25149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Refusing update from remote peer 25d39d64d04b4b71b2e72682026b02a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 03:52:51.559370 25208 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:52:51.560262 24549 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 03:52:51.560724 24549 consensus_peers.cc:487] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 -> Peer 43ce326bae49427c8350369649feafb8 (127.17.54.253:44003): Couldn't send request to peer 43ce326bae49427c8350369649feafb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:52:51.562371 25098 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:51.562491 25098 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:51.562547 25098 tablet_replica.cc:330] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: stopping tablet replica
I20260812 03:52:51.575845 25098 master.cc:552] Master@127.17.54.253:44003 shutdown complete.
I20260812 03:52:51.575927 25098 master_runner.cc:305] Clearing existing system tablet
I20260812 03:52:51.576067 25098 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:51.576143 25098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:51.576828 25098 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:51.577386 25213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.577625 25098 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:51.577733 25098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:51.577777 25098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
uuid: "43ce326bae49427c8350369649feafb8"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "e1a5b4673ed4c5f3f92aef4b827c171c"
server_key_iv: "fa97c003956e3f6e6532082a7e77e408"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.578684 25098 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:51.580701 25098 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:52:51.580770 25098 log.cc:1196] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 03:52:51.581012 25098 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Deleting consensus metadata
I20260812 03:52:51.581176 25098 master_runner.cc:315] Copying system tablet from 127.17.54.254:42117
I20260812 03:52:51.581868 25098 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: tablet copy: Beginning tablet copy session from remote peer at address 127.17.54.254:42117
I20260812 03:52:51.588423 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.582260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:55996)
0812 03:52:51.582429 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.582440 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.582541 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.582692 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.582693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.582879 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.582927 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.582929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.583626 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.583628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.584192 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.584194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.584592 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.584593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.584667 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.585180 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.585186 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.585524 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.587277 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.587278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.587279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.587463 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.587689 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.587690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.587690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.587757 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.588034 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.588035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.588085 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.588192 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.588312 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":124,"threads_started":1}
I20260812 03:52:51.588413 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.582459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.17.54.254:42117)
0812 03:52:51.582534 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.582536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.582548 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:51.582616 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.582617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.582627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.582685 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.583021 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.583539 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.584289 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.584501 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.584590 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.585634 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.585635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.585635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.585644 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.587157 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.587546 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.587548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.587548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.587613 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.587823 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.587824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.587825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.587949 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.588042 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.588221 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.588310 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:52:51.588847 24591 tablet_copy_service.cc:140] P 25d39d64d04b4b71b2e72682026b02a1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 43ce326bae49427c8350369649feafb8 ({username='oryx', principal='oryx/127.17.54.253@KRBTEST.COM'} at 127.0.0.1:55996)
I20260812 03:52:51.588991 24591 tablet_copy_service.cc:161] P 25d39d64d04b4b71b2e72682026b02a1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 43ce326bae49427c8350369649feafb8 at {username='oryx', principal='oryx/127.17.54.253@KRBTEST.COM'} at 127.0.0.1:55996: session id = 43ce326bae49427c8350369649feafb8-00000000000000000000000000000000
I20260812 03:52:51.589756 24591 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:52:51.590543 25098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:52:51.592095 25098 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: tablet copy: Starting download of 0 data blocks...
I20260812 03:52:51.592259 25098 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: tablet copy: Starting download of 1 WAL segments...
I20260812 03:52:51.593377 25098 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:52:51.594341 24591 tablet_copy_service.cc:342] P 25d39d64d04b4b71b2e72682026b02a1: Request end of tablet copy session 43ce326bae49427c8350369649feafb8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.54.253@KRBTEST.COM'} at 127.0.0.1:55996
I20260812 03:52:51.594411 24591 tablet_copy_service.cc:433] P 25d39d64d04b4b71b2e72682026b02a1: ending tablet copy session 43ce326bae49427c8350369649feafb8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 43ce326bae49427c8350369649feafb8
I20260812 03:52:51.597518 25098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:51.599082 25225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.599131 25221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.599228 25223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:51.602504 25098 server_base.cc:1034] running on GCE node
I20260812 03:52:51.602648 25098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:51.602846 25098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:51.604015 25098 hybrid_clock.cc:648] HybridClock initialized: now 1786506771604006 us; error 29 us; skew 500 ppm
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:51.605890 25098 init.cc:377] Logged in from keytab as oryx/127.17.54.253@KRBTEST.COM (short username oryx)
I20260812 03:52:51.606340 25098 webserver.cc:460] Webserver started at http://127.17.54.253:46685/ using document root <none> and password file <none>
I20260812 03:52:51.606487 25098 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:51.606537 25098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:51.607622 25098 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:51.608429 25234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.608649 25098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:51.608731 25098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
uuid: "43ce326bae49427c8350369649feafb8"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "e1a5b4673ed4c5f3f92aef4b827c171c"
server_key_iv: "fa97c003956e3f6e6532082a7e77e408"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.608814 25098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:51.628137 25098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:51.630064 25098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:51.633548 25098 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.253:44003
I20260812 03:52:51.633569 25296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.253:44003 every 8 connection(s)
I20260812 03:52:51.633919 25098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
I20260812 03:52:51.634914 25297 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:52:51.635385 25297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Bootstrap starting.
I20260812 03:52:51.638221 25297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:52:51.638558 25297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Bootstrap complete.
I20260812 03:52:51.638799 25297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } }
I20260812 03:52:51.638899 25297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 43ce326bae49427c8350369649feafb8, State: Initialized, Role: LEARNER
I20260812 03:52:51.638952 25297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } }
I20260812 03:52:51.639178 25297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:51.639171 25300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } }
I20260812 03:52:51.639284 25300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:52:51.640805 25316 catalog_manager.cc:1259] Loaded cluster ID: 7b6f610943c94719ac39e64ebffbf72d
I20260812 03:52:51.640857 25316 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: loading cluster ID for follower catalog manager: success
I20260812 03:52:51.642149 25316 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: acquiring CA information for follower catalog manager: success
I20260812 03:52:51.642474 25316 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:52:51.682085 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.675102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57154 (local address 127.17.54.253:44003)
0812 03:52:51.675285 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.675287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.675293 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:52:51.675317 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.675318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.675327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.675444 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.676096 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.676754 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.677578 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.677783 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.677857 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.679272 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.679274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.679274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.679282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.680837 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.681241 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.681242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.681242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.681308 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.681527 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.681528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.681528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.681650 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.681758 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.681872 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.681966 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for oryx/127.17.54.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.252.keytab.
Entry for principal oryx/127.17.54.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.252.keytab.
I20260812 03:52:51.714835 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.252:34973
--webserver_interface=127.17.54.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.252.keytab
--principal=oryx/127.17.54.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:42117,127.17.54.253:44003,127.17.54.252:34973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:51.805823 25321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:51.806044 25321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:51.806068 25321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:51.807343 25321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:51.807394 25321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:51.807408 25321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:51.807420 25321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:51.807433 25321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:51.809247 25321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:42117,127.17.54.253:44003,127.17.54.252:34973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.252.keytab
--principal=oryx/127.17.54.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.252:34973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.54.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:51.809538 25321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:51.809819 25321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:51.815091 25328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.815097 25329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.815119 25331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:51.815330 25321 server_base.cc:1034] running on GCE node
I20260812 03:52:51.815646 25321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:51.815938 25321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:51.817078 25321 hybrid_clock.cc:648] HybridClock initialized: now 1786506771817065 us; error 32 us; skew 500 ppm
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:51.819981 25321 init.cc:377] Logged in from keytab as oryx/127.17.54.252@KRBTEST.COM (short username oryx)
I20260812 03:52:51.820613 25321 webserver.cc:460] Webserver started at http://127.17.54.252:35731/ using document root <none> and password file <none>
I20260812 03:52:51.820860 25321 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:51.820930 25321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:51.821084 25321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:51.822238 25321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data/instance:
uuid: "3814bf0897484478a2af880fb7544cdf"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "0b9db0b2d4a97834c64dcd8dc9c535ca"
server_key_iv: "c76f8b8d844a1425950eed460598260c"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.822706 25321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal/instance:
uuid: "3814bf0897484478a2af880fb7544cdf"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "0b9db0b2d4a97834c64dcd8dc9c535ca"
server_key_iv: "c76f8b8d844a1425950eed460598260c"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.827104 25321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 03:52:51.828094 25339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.828297 25321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:51.828430 25321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
uuid: "3814bf0897484478a2af880fb7544cdf"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "0b9db0b2d4a97834c64dcd8dc9c535ca"
server_key_iv: "c76f8b8d844a1425950eed460598260c"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.828542 25321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:51.857090 25321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:51.859808 25321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:51.860006 25321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:51.865072 25321 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.252:34973
I20260812 03:52:51.865121 25405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.252:34973 every 8 connection(s)
I20260812 03:52:51.865561 25321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
I20260812 03:52:51.866770 25406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:51.868373 25406 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:51.871280 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 25321
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.252@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.54.252@KRBTEST.COM
I20260812 03:52:51.878990 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.869198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.17.54.254:42117)
0812 03:52:51.869277 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.869280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.869445 (+ 165us) server_negotiation.cc:407] Connection header received
0812 03:52:51.869956 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.869958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.869969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.870058 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.870765 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.871515 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.872762 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.873085 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.873148 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.875873 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.875874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.875875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.875913 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.877427 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.877872 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.877873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.877873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.877942 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.878233 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.878234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.878234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.878414 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.878494 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.878811 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.878923 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:52:51.879148 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.869037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56012)
0812 03:52:51.869296 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.869331 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.869857 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.870133 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.870137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.870467 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.870672 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.870676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.871611 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.871613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.872673 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.872676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.873189 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.873190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.873315 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.873872 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.873882 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.875785 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.877514 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.877520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.877521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.877793 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.878028 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.878029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.878030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.878152 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.878485 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.878491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.878646 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.878769 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.878941 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":102,"threads_started":1}
I20260812 03:52:51.879729 25406 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:51.882570 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.871986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.17.54.252:34973)
0812 03:52:51.872210 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.872215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.872237 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:52:51.872258 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.872259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.872279 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.872367 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.873058 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.874088 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.874892 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.875350 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.875476 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.879278 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.879283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.879284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.879302 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.881149 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.881573 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.881577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.881577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.881641 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.881879 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.881880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.881880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.882050 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.882193 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.882352 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.882441 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":148,"threads_started":1}
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.252@KRBTEST.COM for oryx/127.17.54.253@KRBTEST.COM
I20260812 03:52:51.887377 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.880064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.0.0.1:57164)
0812 03:52:51.880124 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.880136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.880208 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.880368 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.880369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.880509 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.880571 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.880573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.881554 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.881556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.882411 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.882414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.882926 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.882927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.882984 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.883423 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.883427 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.884737 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.886467 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.886468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.886468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.886639 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.886829 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.886829 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.886830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.886881 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.887147 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.887147 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.887184 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.887236 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.887313 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 03:52:51.887462 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.880152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.17.54.253:44003)
0812 03:52:51.880226 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.880228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.880239 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:51.880270 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.880272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.880290 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.880353 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.880674 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.881441 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.882498 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.882847 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.882924 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.884870 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.884871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.884871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.884879 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.886379 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.886713 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.886713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.886713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.886767 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.886963 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.886964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.886965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.887083 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.887146 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.887274 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.887350 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:52:51.887847 25406 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.252@KRBTEST.COM for oryx/127.17.54.252@KRBTEST.COM
I20260812 03:52:51.893936 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.888157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.0.0.1:55902)
0812 03:52:51.888210 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.888222 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.888265 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.888358 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.888359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.888466 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.888501 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.888502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.889102 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.889103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.889583 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.889585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.889909 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.889910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.889945 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.890353 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.890358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.891464 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.892908 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.892909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.892909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.893082 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.893254 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.893255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.893255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.893304 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.893673 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.893674 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.893716 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.893784 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.893868 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:52:51.894001 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.888216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55902 (local address 127.17.54.252:34973)
0812 03:52:51.888274 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.888276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.888284 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:51.888302 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.888303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.888312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.888356 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.888563 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.889035 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.889655 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.889847 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.889918 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.891554 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.891555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.891555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.891561 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.892831 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.893147 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.893148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.893148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.893198 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.893423 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.893424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.893424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.893572 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.893652 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.893818 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.893920 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":42}
I20260812 03:52:51.895473 25406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Bootstrap starting.
I20260812 03:52:51.896337 25406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:51.896637 25406 log.cc:824] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:51.897454 25406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: No bootstrap required, opened a new log
I20260812 03:52:51.899055 25406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } }
I20260812 03:52:51.899181 25406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:51.899204 25406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3814bf0897484478a2af880fb7544cdf, State: Initialized, Role: FOLLOWER
I20260812 03:52:51.899353 25406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } }
I20260812 03:52:51.899773 25413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } } }
I20260812 03:52:51.899845 25406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:51.899860 25413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:52:51.901592 25427 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:51.901650 25427 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:51.904100 25321 master_runner.cc:428] Detected that this master 3814bf0897484478a2af880fb7544cdf is joining an existing cluster
I20260812 03:52:51.904156 25321 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.54.252:34973
I20260812 03:52:51.912104 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.905605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56026 (local address 127.17.54.254:42117)
0812 03:52:51.905657 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.905659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.905666 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:52:51.905689 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.905690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.905701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.905756 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.906185 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.906767 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.907589 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.907811 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.907890 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.909094 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.909095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.909095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.909103 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.910870 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.911266 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.911268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.911268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.911332 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.911554 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.911555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.911555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.911676 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.911751 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.911920 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.912001 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:52:51.912127 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.905144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56026)
0812 03:52:51.905539 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.905551 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.905614 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.905769 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.905770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.906029 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.906102 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.906105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.906856 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.906857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.907437 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.907439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.907924 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.907925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.907992 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.908530 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.908535 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.909000 (+ 465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.910961 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.910962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.910963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.911186 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.911409 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.911410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.911410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.911483 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.911744 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.911745 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.911789 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.911894 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.912002 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":91,"threads_started":1}
I20260812 03:52:51.913594 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.905203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57170 (local address 127.17.54.253:44003)
0812 03:52:51.905257 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.905259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.905753 (+ 494us) server_negotiation.cc:407] Connection header received
0812 03:52:51.905840 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.905841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.905853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.905902 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.906372 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.907136 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.908395 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.908714 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.908791 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.909862 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.909864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.909864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.909873 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.911664 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.912762 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.912763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.912764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.912826 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.913027 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.913028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.913029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.913151 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.913190 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.913399 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.913481 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:52:51.913941 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.905173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.0.0.1:57170)
0812 03:52:51.905684 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.905694 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.905781 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.905930 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.905931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.906180 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.906236 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.906238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.907238 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.907239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.908226 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.908229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.908808 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.908809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.908882 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.909307 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.909312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.909714 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.912354 (+ 2640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.912355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.912363 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.912667 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.912899 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.912900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.912900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.912959 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.913231 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.913232 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.913282 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.913718 (+ 436us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.913829 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":96,"threads_started":1}
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506770, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.254@KRBTEST.COM for oryx/127.17.54.252@KRBTEST.COM
I20260812 03:52:51.933962 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 24640
I20260812 03:52:51.934471 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.914907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.0.0.1:55916)
0812 03:52:51.914998 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.915008 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.915045 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.915485 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.915486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.915654 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.915701 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.915703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.916356 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.916357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.917196 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.917198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.917564 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.917565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.917637 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.918023 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.918028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.919510 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.933111 (+ 13601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.933112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.933113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.933358 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.933619 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.933620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.933620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.933694 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.934045 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.934047 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.934104 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.934155 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.934289 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 03:52:51.936107 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.915110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.17.54.252:34973)
0812 03:52:51.915326 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.915327 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.915333 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:52:51.915378 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.915379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.915395 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.915452 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.915803 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.916273 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.917280 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.917489 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.917542 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.919647 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.919648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.919648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.919655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.932989 (+ 13334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.933446 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.933448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.933448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.933518 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.933771 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.933771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.933772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.933928 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.935736 (+ 1808us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.935792 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.935910 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":135,"threads_started":1}
I20260812 03:52:51.938954 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.905441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.17.54.252:34973)
0812 03:52:51.905476 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.905478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.912273 (+ 6795us) server_negotiation.cc:407] Connection header received
0812 03:52:51.912337 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.912338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.912349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.912397 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.912787 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.913489 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.914835 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.915035 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.915118 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.931960 (+ 16842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.931962 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.931962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.931970 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.934074 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.937983 (+ 3909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.937985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.937985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.938075 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.938345 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.938346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.938347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.938519 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.938587 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.938747 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.938832 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:52:51.939257 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.905418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.0.0.1:55908)
0812 03:52:51.912218 (+ 6800us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.912230 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.912283 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.912431 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.912432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.912653 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.912710 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.912712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.913568 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.913569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.914773 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.914776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.915116 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.915117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.915198 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.930993 (+ 15795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.930999 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.931580 (+ 581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.937625 (+ 6045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.937626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.937627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.937873 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.938162 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.938163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.938164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.938232 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.938611 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.938612 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.938650 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.939031 (+ 381us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.939119 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6787,"thread_start_us":73,"threads_started":1}
I20260812 03:52:51.941409 24564 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.54.252:34973: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3814bf0897484478a2af880fb7544cdf"
member_type: NON_VOTER
last_known_addr {
host: "127.17.54.252"
port: 34973
}
attrs {
promote: true
}
}
dest_uuid: "25d39d64d04b4b71b2e72682026b02a1"
cas_config_opid_index: 5
I20260812 03:52:51.941720 24564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } }
I20260812 03:52:51.942269 25209 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3814bf0897484478a2af880fb7544cdf (127.17.54.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.942982 25373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:51.943373 25209 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.54.252:34973
I20260812 03:52:51.943462 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.943529 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:51.944156 25373 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 FOLLOWER]: Refusing update from remote peer 25d39d64d04b4b71b2e72682026b02a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:52:51.944450 25208 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:51.946676 25373 raft_consensus.cc:858] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } }
W20260812 03:52:51.946762 25373 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } }
I20260812 03:52:51.946954 25321 master.cc:537] Master@127.17.54.252:34973 shutting down...
I20260812 03:52:51.947299 25373 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 43ce326bae49427c8350369649feafb8 (127.17.54.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } }
I20260812 03:52:51.948415 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.943727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.0.0.1:57180)
0812 03:52:51.944177 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.944188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.944240 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.944530 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.944531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.944532 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:51.944584 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.944586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.945675 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.945676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.947076 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.947080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.948144 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.948145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.948217 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.948243 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.948297 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.948341 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430}
I20260812 03:52:51.948459 25413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25d39d64d04b4b71b2e72682026b02a1. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } }
I20260812 03:52:51.948552 25413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:52:51.948589 25413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } }
I20260812 03:52:51.948642 25413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:52:51.948673 25413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } }
I20260812 03:52:51.948725 25413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:52:51.948927 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.943837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.17.54.253:44003)
0812 03:52:51.943881 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.943883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.944296 (+ 413us) server_negotiation.cc:407] Connection header received
0812 03:52:51.944328 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.944329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.944356 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.944414 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:51.944657 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.945466 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.947237 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.947808 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.947892 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.947974 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.948796 (+ 822us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.948847 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:52:51.949150 25267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 LEARNER]: Refusing update from remote peer 25d39d64d04b4b71b2e72682026b02a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:52:51.949340 25208 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 03:52:51.949935 25267 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3814bf0897484478a2af880fb7544cdf (127.17.54.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.950191 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:46153
--local_ip_for_outbound_sockets=127.17.54.193
--tserver_master_addrs=127.17.54.254:42117
--webserver_port=39737
--webserver_interface=127.17.54.193
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab
--principal=oryx/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003,127.17.54.252:34973 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:52:51.951066 25300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25d39d64d04b4b71b2e72682026b02a1. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.951153 25300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:52:51.951210 25300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.951251 25300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:52:51.951421 25208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: NON_VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: true } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.951534 25208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:51.951798 25438 mvcc.cc:205] Tried to move back new op lower bound from 7317531733309345792 to 7317531733238358016. Current Snapshot: MvccSnapshot[applied={T|T < 7317531733250838528}]
W20260812 03:52:51.952052 24548 consensus_peers.cc:487] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 -> Peer 3814bf0897484478a2af880fb7544cdf (127.17.54.252:34973): Couldn't send request to peer 3814bf0897484478a2af880fb7544cdf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:52:51.952598 25321 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260812 03:52:51.952685 25321 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260812 03:52:51.952723 25321 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: stopping tablet replica
I20260812 03:52:51.953547 25209 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: attempting to promote NON_VOTER 43ce326bae49427c8350369649feafb8 to VOTER
I20260812 03:52:51.953752 25209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } }
I20260812 03:52:51.954348 25267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 LEARNER]: Refusing update from remote peer 25d39d64d04b4b71b2e72682026b02a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 03:52:51.954525 25208 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 03:52:51.955338 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.955127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.0.0.1:55926)
0812 03:52:51.955255 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.955301 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:34973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260812 03:52:51.955427 24548 consensus_peers.cc:487] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 -> Peer 3814bf0897484478a2af880fb7544cdf (127.17.54.252:34973): Couldn't send request to peer 3814bf0897484478a2af880fb7544cdf. Status: Network error: Client connection negotiation failed: client connection to 127.17.54.252:34973: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:52:51.956188 25208 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 43ce326bae49427c8350369649feafb8 (127.17.54.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.957406 25208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.957507 25208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:51.957669 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.957755 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:51.957839 25267 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 43ce326bae49427c8350369649feafb8 (127.17.54.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.959107 25444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:51.959226 25444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:51.967352 25321 master.cc:552] Master@127.17.54.252:34973 shutdown complete.
I20260812 03:52:51.967406 25321 master_runner.cc:305] Clearing existing system tablet
I20260812 03:52:51.967542 25321 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:51.967583 25321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:51.968333 25321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:51.968894 25453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.969013 25321 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:51.969059 25321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:51.969080 25321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
uuid: "3814bf0897484478a2af880fb7544cdf"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "0b9db0b2d4a97834c64dcd8dc9c535ca"
server_key_iv: "c76f8b8d844a1425950eed460598260c"
server_key_version: "encryptionkey@0"
I20260812 03:52:51.969800 25321 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:51.971511 25321 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:52:51.971585 25321 log.cc:1196] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 03:52:51.971829 25321 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Deleting consensus metadata
I20260812 03:52:51.971956 25321 master_runner.cc:315] Copying system tablet from 127.17.54.254:42117
I20260812 03:52:51.972577 25321 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: tablet copy: Beginning tablet copy session from remote peer at address 127.17.54.254:42117
I20260812 03:52:51.980394 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.973081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.17.54.254:42117)
0812 03:52:51.973127 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:51.973130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:51.973141 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:51.973197 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:51.973198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:51.973210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:51.973273 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:51.973643 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.974523 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.975585 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.975908 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.975982 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.977148 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:51.977167 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:51.977168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:51.977176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:51.979089 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.979499 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.979500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.979501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.979599 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:51.979817 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:51.979819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:51.979819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:51.979973 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:51.980046 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:51.980222 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 03:52:51.980303 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:52:51.980396 25461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:51.972889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56036)
0812 03:52:51.973053 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:51.973067 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:51.973137 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:51.973293 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:51.973295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:51.973491 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:51.973552 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.973555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.974620 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.974621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.975445 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:51.975448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:51.975990 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:51.975991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:51.976069 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:51.976621 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:51.976627 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:51.977041 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:51.979185 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.979186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.979187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.979412 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.979677 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:51.979678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:51.979678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:51.979747 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:51.980046 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:51.980047 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:51.980100 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:52:51.980175 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:52:51.980290 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260812 03:52:51.981035 24591 tablet_copy_service.cc:140] P 25d39d64d04b4b71b2e72682026b02a1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3814bf0897484478a2af880fb7544cdf ({username='oryx', principal='oryx/127.17.54.252@KRBTEST.COM'} at 127.0.0.1:56036)
I20260812 03:52:51.981101 24591 tablet_copy_service.cc:161] P 25d39d64d04b4b71b2e72682026b02a1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3814bf0897484478a2af880fb7544cdf at {username='oryx', principal='oryx/127.17.54.252@KRBTEST.COM'} at 127.0.0.1:56036: session id = 3814bf0897484478a2af880fb7544cdf-00000000000000000000000000000000
I20260812 03:52:51.981729 24591 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:52:51.982244 25321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:52:51.983573 25321 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: tablet copy: Starting download of 0 data blocks...
I20260812 03:52:51.983728 25321 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: tablet copy: Starting download of 1 WAL segments...
I20260812 03:52:51.984602 25321 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:52:51.985570 24591 tablet_copy_service.cc:342] P 25d39d64d04b4b71b2e72682026b02a1: Request end of tablet copy session 3814bf0897484478a2af880fb7544cdf-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.54.252@KRBTEST.COM'} at 127.0.0.1:56036
I20260812 03:52:51.985626 24591 tablet_copy_service.cc:433] P 25d39d64d04b4b71b2e72682026b02a1: ending tablet copy session 3814bf0897484478a2af880fb7544cdf-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3814bf0897484478a2af880fb7544cdf
I20260812 03:52:51.988214 25321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:51.989414 25462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:51.989509 25465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:51.990363 25321 server_base.cc:1034] running on GCE node
W20260812 03:52:51.998484 25463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:51.998679 25321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:51.998848 25321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:51.999981 25321 hybrid_clock.cc:648] HybridClock initialized: now 1786506771999960 us; error 47 us; skew 500 ppm
Aug 12 03:52:51 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506771, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:52.001644 25321 init.cc:377] Logged in from keytab as oryx/127.17.54.252@KRBTEST.COM (short username oryx)
I20260812 03:52:52.001998 25321 webserver.cc:460] Webserver started at http://127.17.54.252:39129/ using document root <none> and password file <none>
I20260812 03:52:52.002130 25321 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:52.002173 25321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:52.003047 25321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:52.003700 25474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.003814 25321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:52:52.003849 25321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
uuid: "3814bf0897484478a2af880fb7544cdf"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "0b9db0b2d4a97834c64dcd8dc9c535ca"
server_key_iv: "c76f8b8d844a1425950eed460598260c"
server_key_version: "encryptionkey@0"
I20260812 03:52:52.003912 25321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:52.036315 25321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:52.038877 25321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:52.042871 25321 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.252:34973
I20260812 03:52:52.042923 25532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.252:34973 every 8 connection(s)
I20260812 03:52:52.043287 25321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
I20260812 03:52:52.044405 25533 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:52:52.045028 25533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Bootstrap starting.
I20260812 03:52:52.047909 25533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:52:52.048177 25533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: Bootstrap complete.
I20260812 03:52:52.048432 25533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } }
I20260812 03:52:52.048540 25533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3814bf0897484478a2af880fb7544cdf, State: Initialized, Role: LEARNER
I20260812 03:52:52.048583 25533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } }
I20260812 03:52:52.048822 25533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:52.048835 25536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:52.049134 25536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: LEARNER
I20260812 03:52:52.051023 25550 catalog_manager.cc:1259] Loaded cluster ID: 7b6f610943c94719ac39e64ebffbf72d
I20260812 03:52:52.051148 25550 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: loading cluster ID for follower catalog manager: success
I20260812 03:52:52.052740 25550 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: acquiring CA information for follower catalog manager: success
I20260812 03:52:52.053138 25550 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 03:52:52.062660 25448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:52.062901 25448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:52.062929 25448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:52.064495 25448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:52.064555 25448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:52.064587 25448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:52.066514 25448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab
--principal=oryx/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:46153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=39737
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003,127.17.54.252:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:52.066818 25448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:52.067118 25448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:52.072642 25557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:52.072638 25555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:52.072778 25448 server_base.cc:1034] running on GCE node
W20260812 03:52:52.072638 25559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:52.073148 25448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:52.073410 25448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:52.074589 25448 hybrid_clock.cc:648] HybridClock initialized: now 1786506772074575 us; error 34 us; skew 500 ppm
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:52.077294 25448 init.cc:377] Logged in from keytab as oryx/127.17.54.193@KRBTEST.COM (short username oryx)
I20260812 03:52:52.077811 25448 webserver.cc:460] Webserver started at http://127.17.54.193:39737/ using document root <none> and password file <none>
I20260812 03:52:52.078078 25448 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:52.078161 25448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:52.079749 25448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:52.080497 25568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.080718 25448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:52.080834 25448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "b2c3658deb024df3891ff955776281c4"
format_stamp: "Formatted at 2026-08-12 03:52:50 on dist-test-slave-tm1g"
server_key: "c29e5ed3714b55d2824a722820644b46"
server_key_iv: "802491f5ae8717dd631ffe497c053fc3"
server_key_version: "encryptionkey@0"
I20260812 03:52:52.081214 25448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:52.120242 25448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:52.122829 25448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:52.122978 25448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:52.123271 25448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:52.123670 25448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:52.123728 25448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.123771 25448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:52.123799 25448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.130162 25448 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:46153
I20260812 03:52:52.130244 25700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:46153 every 8 connection(s)
I20260812 03:52:52.130645 25448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:52.138569 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 25448
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.253@KRBTEST.COM
I20260812 03:52:52.138819 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 24789
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.252@KRBTEST.COM
I20260812 03:52:52.144024 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.131492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:35955 (local address 127.17.54.254:42117)
0812 03:52:52.131550 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.131553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.131849 (+ 296us) server_negotiation.cc:407] Connection header received
0812 03:52:52.133042 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.133043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.133056 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.133112 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.134031 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.134848 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136212 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.136424 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136518 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.140707 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.140708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.140709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.140718 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.142591 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.142999 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.143000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.143000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.143076 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.143282 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.143283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.143283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.143445 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.143515 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.143883 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.143957 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:52:52.144361 25706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.131524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.193:35955)
0812 03:52:52.131712 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.131757 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.132845 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.133180 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.133184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.133691 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.133909 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.133914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.134965 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.134965 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.136109 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.136111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.136576 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.136643 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.137364 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.137369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.140580 (+ 3211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.142678 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.142679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.142680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.142919 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.143148 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.143149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.143149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.143218 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.143517 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.143525 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.143718 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.143842 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.144053 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":91,"thread_start_us":95,"threads_started":1}
I20260812 03:52:52.144838 25702 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42117
I20260812 03:52:52.144960 25702 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.145160 25702 heartbeater.cc:511] Master 127.17.54.254:42117 requested a full tablet report, sending...
I20260812 03:52:52.145702 24564 ts_manager.cc:194] Re-registered known tserver with Master: b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153)
I20260812 03:52:52.146258 24564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:35955
I20260812 03:52:52.146884 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.131478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:47821 (local address 127.17.54.253:44003)
0812 03:52:52.131531 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.131534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.131851 (+ 317us) server_negotiation.cc:407] Connection header received
0812 03:52:52.132973 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.132975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.132989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.133078 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.134031 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.134830 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136162 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.136467 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136529 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.139245 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.139247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.139247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.139256 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.141221 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.141795 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.141796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.141797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.141871 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.142148 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.142150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.142150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.142291 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.142397 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.146695 (+ 4298us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.146789 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:52:52.147517 25707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.131524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.17.54.193:47821)
0812 03:52:52.131747 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.131787 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.132852 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.133180 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.133184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.133691 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.133909 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.133914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.134936 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.134936 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.136081 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.136084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.136580 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.136648 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.137421 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.137425 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.139123 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.141345 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.141354 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.141356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.141679 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.141940 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.141941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.141942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.142075 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.146510 (+ 4435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.146512 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.146557 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.146656 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.146788 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":69,"spinlock_wait_cycles":7040,"thread_start_us":135,"threads_started":1}
I20260812 03:52:52.147949 25703 heartbeater.cc:347] Connected to a master server at 127.17.54.253:44003
I20260812 03:52:52.147997 25703 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.148069 25703 heartbeater.cc:511] Master 127.17.54.253:44003 requested a full tablet report, sending...
I20260812 03:52:52.148306 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:42359
--local_ip_for_outbound_sockets=127.17.54.194
--tserver_master_addrs=127.17.54.254:42117
--webserver_port=44695
--webserver_interface=127.17.54.194
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab
--principal=oryx/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003,127.17.54.252:34973 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:52:52.148525 25254 ts_manager.cc:194] Registered new tserver with Master: b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153)
I20260812 03:52:52.150697 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.131835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.17.54.193:46085)
0812 03:52:52.131966 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.131976 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.132857 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.133277 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.133279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.133690 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.133908 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.133912 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.134914 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.134915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.136004 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.136007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.136486 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.136640 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.137315 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.137327 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.147281 (+ 9954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.149530 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.149531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.149532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.149772 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.150033 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.150034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.150035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.150097 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.150399 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.150401 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.150451 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.150510 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.150608 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":82,"thread_start_us":90,"threads_started":1}
I20260812 03:52:52.150889 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.131918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:46085 (local address 127.17.54.252:34973)
0812 03:52:52.132033 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.132035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.132044 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:52.133156 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.133157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.133170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.133256 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.134009 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.134808 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136096 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.136394 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.136462 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.147388 (+ 10926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.147390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.147391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.147401 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.149440 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.149859 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.149860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.149860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.149956 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.150162 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.150163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.150163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.150337 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.150412 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.150560 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.150665 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260812 03:52:52.151338 25701 heartbeater.cc:347] Connected to a master server at 127.17.54.252:34973
I20260812 03:52:52.151393 25701 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.151455 25701 heartbeater.cc:511] Master 127.17.54.252:34973 requested a full tablet report, sending...
I20260812 03:52:52.151919 25484 ts_manager.cc:194] Registered new tserver with Master: b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153)
W20260812 03:52:52.235044 25712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:52.235261 25712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:52.235286 25712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:52.236431 25712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:52.236469 25712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:52.236495 25712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:52.238010 25712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab
--principal=oryx/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:42359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=44695
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003,127.17.54.252:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:52.238219 25712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:52.238559 25712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:52.244230 25717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:52.244230 25722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:52.244410 25720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:52.244681 25712 server_base.cc:1034] running on GCE node
I20260812 03:52:52.244918 25712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:52.245157 25712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:52.246371 25712 hybrid_clock.cc:648] HybridClock initialized: now 1786506772246352 us; error 31 us; skew 500 ppm
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:52.252521 25712 init.cc:377] Logged in from keytab as oryx/127.17.54.194@KRBTEST.COM (short username oryx)
I20260812 03:52:52.253118 25712 webserver.cc:460] Webserver started at http://127.17.54.194:44695/ using document root <none> and password file <none>
I20260812 03:52:52.253401 25712 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:52.253500 25712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:52.255096 25712 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:52:52.255790 25730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.255996 25712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:52.256107 25712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "9b21f85a09e64588806be1c1e5ce1c79"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "565a7ec4d949f69dc84270f946d7b167"
server_key_iv: "27939f32c19e65717ea3050484f5b0cc"
server_key_version: "encryptionkey@0"
I20260812 03:52:52.256484 25712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:52.289340 25712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:52.291600 25712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:52.291785 25712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:52.292094 25712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:52.292448 25712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:52.292502 25712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.292543 25712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:52.292572 25712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.299225 25712 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:42359
I20260812 03:52:52.299273 25861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:42359 every 8 connection(s)
I20260812 03:52:52.299642 25712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:52:52.305444 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 25712
I20260812 03:52:52.305637 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 24941
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:52.314546 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.300851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:45193 (local address 127.17.54.254:42117)
0812 03:52:52.300924 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.300926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.301090 (+ 164us) server_negotiation.cc:407] Connection header received
0812 03:52:52.302051 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.302053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.302068 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.302132 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.303123 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.303661 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.304850 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.305047 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.305142 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.310586 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.310588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.310588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.310596 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.312442 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.313024 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.313025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.313026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.313091 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.313400 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.313401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.313402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.313568 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.313657 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.314347 (+ 690us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.314442 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:52:52.315411 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.300780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.194:45193)
0812 03:52:52.301050 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.301057 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.301923 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.302280 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.302284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.302791 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.303013 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.303018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.303780 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.303781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.304766 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.304768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.305167 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.305168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.305331 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.308037 (+ 2706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.308050 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.310463 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.312555 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.312564 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.312566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.312908 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.313163 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.313165 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.313165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.313311 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.313661 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.313669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.313851 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.314666 (+ 815us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.315148 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":13,"thread_start_us":117,"threads_started":1}
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.253@KRBTEST.COM
I20260812 03:52:52.316480 25863 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42117
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.252@KRBTEST.COM
I20260812 03:52:52.316596 25863 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.316815 25863 heartbeater.cc:511] Master 127.17.54.254:42117 requested a full tablet report, sending...
I20260812 03:52:52.319258 24564 ts_manager.cc:194] Re-registered known tserver with Master: 9b21f85a09e64588806be1c1e5ce1c79 (127.17.54.194:42359)
I20260812 03:52:52.319566 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:42841
--local_ip_for_outbound_sockets=127.17.54.195
--tserver_master_addrs=127.17.54.254:42117
--webserver_port=40837
--webserver_interface=127.17.54.195
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab
--principal=oryx/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003,127.17.54.252:34973 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:52:52.319867 24564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:45193
I20260812 03:52:52.321071 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.300447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.17.54.194:52597)
0812 03:52:52.300950 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.300963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.301942 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.302280 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.302284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.302791 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.303013 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.303018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.312233 (+ 9215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.312235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.313195 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.313197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.313735 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.313736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.313802 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.314254 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.314260 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.316524 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.319852 (+ 3328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.319853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.319854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.320118 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.320385 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.320386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.320387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.320470 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.320803 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.320803 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.320846 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.320903 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.320992 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":4,"thread_start_us":97,"threads_started":1}
I20260812 03:52:52.321249 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.300538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:52597 (local address 127.17.54.253:44003)
0812 03:52:52.300582 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.300584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.301036 (+ 452us) server_negotiation.cc:407] Connection header received
0812 03:52:52.302107 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.302108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.302125 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.302179 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.304004 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.304790 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.313302 (+ 8512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.313646 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.313738 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.316724 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.316725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.316726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.316735 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.319750 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.320226 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.320228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.320228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.320305 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.320557 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.320558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.320559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.320727 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.320800 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.320972 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.321093 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:52:52.321704 25865 heartbeater.cc:347] Connected to a master server at 127.17.54.253:44003
I20260812 03:52:52.321765 25865 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.321864 25865 heartbeater.cc:511] Master 127.17.54.253:44003 requested a full tablet report, sending...
I20260812 03:52:52.322278 25254 ts_manager.cc:194] Registered new tserver with Master: 9b21f85a09e64588806be1c1e5ce1c79 (127.17.54.194:42359)
I20260812 03:52:52.323701 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.300447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.17.54.194:48811)
0812 03:52:52.300631 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.300682 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.301913 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.302279 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.302284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.302795 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.303011 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.303016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.303969 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.303971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.304920 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.304923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.305326 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.305327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.305404 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.314404 (+ 9000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.314410 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.316824 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.322544 (+ 5720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.322545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.322546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.322719 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.322954 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.322955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.322955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.323007 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.323318 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.323318 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.323352 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.323405 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.323493 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260812 03:52:52.323695 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.300471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:48811 (local address 127.17.54.252:34973)
0812 03:52:52.300526 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.300529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.300778 (+ 249us) server_negotiation.cc:407] Connection header received
0812 03:52:52.302107 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.302108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.302122 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.302181 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.303120 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.303779 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.305011 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.305237 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.305327 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.316962 (+ 11635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.316964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.316965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.316975 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.322446 (+ 5471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.322802 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.322804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.322804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.322888 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.323083 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.323084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.323084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.323249 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.323326 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.323435 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.323543 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:52.324148 25862 heartbeater.cc:347] Connected to a master server at 127.17.54.252:34973
I20260812 03:52:52.324216 25862 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.324280 25862 heartbeater.cc:511] Master 127.17.54.252:34973 requested a full tablet report, sending...
I20260812 03:52:52.324602 25484 ts_manager.cc:194] Registered new tserver with Master: 9b21f85a09e64588806be1c1e5ce1c79 (127.17.54.194:42359)
W20260812 03:52:52.414976 25870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:52.415202 25870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:52.415231 25870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:52.416663 25870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:52.416716 25870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:52.416747 25870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:52.418505 25870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab
--principal=oryx/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:42841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=40837
--tserver_master_addrs=127.17.54.254:42117,127.17.54.253:44003,127.17.54.252:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:52.418797 25870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:52.419088 25870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:52.426003 25880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:52.426138 25870 server_base.cc:1034] running on GCE node
W20260812 03:52:52.426208 25876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:52.426023 25875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:52.426652 25870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:52.426991 25870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:52.428184 25870 hybrid_clock.cc:648] HybridClock initialized: now 1786506772428154 us; error 46 us; skew 500 ppm
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:52.431208 25870 init.cc:377] Logged in from keytab as oryx/127.17.54.195@KRBTEST.COM (short username oryx)
I20260812 03:52:52.431939 25870 webserver.cc:460] Webserver started at http://127.17.54.195:40837/ using document root <none> and password file <none>
I20260812 03:52:52.432242 25870 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:52.432322 25870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:52.433607 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.430209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.0.0.1:55940)
0812 03:52:52.430296 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.430329 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.430393 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.430626 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.430627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.430629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:52.430692 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.430695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.431478 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.431479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.432638 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.432643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.433319 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.433320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.433390 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.433395 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.433466 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.433527 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 03:52:52.433638 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.430374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55940 (local address 127.17.54.252:34973)
0812 03:52:52.430447 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.430449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.430460 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:52.430510 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.430511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.430536 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.430596 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:52.430768 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.431347 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.432756 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.433218 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.433287 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.433380 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.433540 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.433587 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:52:52.433976 25504 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 03:52:52.434149 25870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:52:52.434149 25536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25d39d64d04b4b71b2e72682026b02a1. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:52.434252 25536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: LEARNER
I20260812 03:52:52.434937 25888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.435098 25870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:52.435209 25870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "b801635b38ab4450b56f8e93d80d5a78"
format_stamp: "Formatted at 2026-08-12 03:52:51 on dist-test-slave-tm1g"
server_key: "6546f1809740d448d093a7bf4d143a65"
server_key_iv: "c27588e43a1bdf32084eee533ed0cb3d"
server_key_version: "encryptionkey@0"
I20260812 03:52:52.435599 25870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:52.453225 25870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:52.455379 25870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:52.455564 25870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:52.455850 25870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:52.456190 25870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:52.456245 25870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.456290 25870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:52.456324 25870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:52.463078 25870 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:42841
I20260812 03:52:52.463192 26015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:42841 every 8 connection(s)
I20260812 03:52:52.463536 25870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:52.467216 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 25870
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.252@KRBTEST.COM
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
Aug 12 03:52:52 dist-test-slave-tm1g krb5kdc[24518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506772, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.253@KRBTEST.COM
I20260812 03:52:52.477628 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.464385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.17.54.195:58079)
0812 03:52:52.464590 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.464628 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.465439 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.465778 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.465779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.466191 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.466465 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.466470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.467496 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.467498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.468957 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.468960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.469583 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.469584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.469800 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.470535 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.470548 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.473697 (+ 3149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.475712 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.475723 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.475726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.476073 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.476303 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.476304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.476304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.476459 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.476788 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.476796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.476984 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.477134 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.477358 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":3,"thread_start_us":125,"threads_started":1}
I20260812 03:52:52.477979 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.464365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:58079 (local address 127.17.54.252:34973)
0812 03:52:52.464413 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.464415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.464723 (+ 308us) server_negotiation.cc:407] Connection header received
0812 03:52:52.465599 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.465601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.465612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.465665 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.466569 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.467190 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.469079 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.469430 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.469477 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.473795 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.473796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.473797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.473805 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.475623 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.476165 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.476166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.476167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.476233 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.476535 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.476536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.476536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.476691 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.476760 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.477779 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.477886 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:52:52.478031 26021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.464701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.195:50299)
0812 03:52:52.464829 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.464841 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.465448 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.465760 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.465765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.466285 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.466465 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.466471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.468590 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.468591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.469502 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.469504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.469951 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.469953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.470025 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.470662 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.470668 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.473279 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.476713 (+ 3434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.476714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.476714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.476989 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.477220 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.477221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.477221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.477287 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.477617 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.477618 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.477670 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.477748 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.477895 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260812 03:52:52.478389 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.464769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:50299 (local address 127.17.54.254:42117)
0812 03:52:52.464825 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.464827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.464906 (+ 79us) server_negotiation.cc:407] Connection header received
0812 03:52:52.465574 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.465576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.465591 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.465665 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.467319 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.468078 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.469594 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.469889 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.469952 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.473398 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.473400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.473400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.473409 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.476611 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.477076 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.477078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.477078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.477149 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.477366 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.477367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.477367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.477517 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.477584 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.478160 (+ 576us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.478267 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:52:52.479391 26017 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42117
I20260812 03:52:52.479547 26017 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.479786 26017 heartbeater.cc:511] Master 127.17.54.254:42117 requested a full tablet report, sending...
I20260812 03:52:52.479919 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.465027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:45089 (local address 127.17.54.253:44003)
0812 03:52:52.465094 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.465096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.465709 (+ 613us) server_negotiation.cc:407] Connection header received
0812 03:52:52.465806 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.465808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.465822 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.465880 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.468197 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.468843 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.469685 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.470001 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.470037 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.475379 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.475381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.475381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.475390 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.477165 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.479018 (+ 1853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.479020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.479020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.479088 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.479280 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.479281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.479281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.479409 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.479471 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.479718 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.479806 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:52:52.480294 24564 ts_manager.cc:194] Re-registered known tserver with Master: b801635b38ab4450b56f8e93d80d5a78 (127.17.54.195:42841)
I20260812 03:52:52.480414 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.464922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.17.54.195:45089)
0812 03:52:52.465631 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.465658 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.465734 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.465913 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.465915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.466220 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:52.466465 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.466471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.468933 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.468934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.469584 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.469587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.470248 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.470249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.470353 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.470872 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:52.470879 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:52.475247 (+ 4368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:52.477246 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.477247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.477248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.478918 (+ 1670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.479162 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:52.479163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:52.479163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:52.479224 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:52.479502 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:52.479503 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:52.479552 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.479664 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.479783 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260812 03:52:52.480891 24564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:50299
I20260812 03:52:52.480968 26018 heartbeater.cc:347] Connected to a master server at 127.17.54.253:44003
I20260812 03:52:52.481026 26018 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.481113 26018 heartbeater.cc:511] Master 127.17.54.253:44003 requested a full tablet report, sending...
I20260812 03:52:52.479391 26016 heartbeater.cc:347] Connected to a master server at 127.17.54.252:34973
I20260812 03:52:52.481281 26016 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:52.481395 26016 heartbeater.cc:511] Master 127.17.54.252:34973 requested a full tablet report, sending...
I20260812 03:52:52.481593 25254 ts_manager.cc:194] Registered new tserver with Master: b801635b38ab4450b56f8e93d80d5a78 (127.17.54.195:42841)
I20260812 03:52:52.481734 25484 ts_manager.cc:194] Registered new tserver with Master: b801635b38ab4450b56f8e93d80d5a78 (127.17.54.195:42841)
I20260812 03:52:52.486268 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.468551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.17.54.252:34973)
0812 03:52:52.471080 (+ 2529us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.471082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.471098 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:52:52.471122 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.471123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.471137 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.471393 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.472322 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.473221 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.478371 (+ 5150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.478683 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.479049 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.482399 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.482401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.482401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.482411 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.484338 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.485212 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.485214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.485215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.485304 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.485579 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.485580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.485581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.485767 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.485913 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.486046 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.486137 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2504,"thread_start_us":56,"threads_started":1}
I20260812 03:52:52.487300 26030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.468347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.17.54.254:42117)
0812 03:52:52.470422 (+ 2075us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.470425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.470434 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:52.471251 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.471252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.471266 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.471798 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.472311 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.476406 (+ 4095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.479320 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.479660 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.479970 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.483556 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.483558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.483558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.483568 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.485641 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.486110 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.486111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.486111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.486179 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.486635 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.486636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.486636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.486800 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.486917 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.487033 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.487164 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2051,"thread_start_us":60,"threads_started":1}
I20260812 03:52:52.487953 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.470528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.17.54.253:44003)
0812 03:52:52.470815 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.470817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.472273 (+ 1456us) server_negotiation.cc:407] Connection header received
0812 03:52:52.472461 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.472462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.472476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.472532 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.473466 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.474421 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.476909 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.477296 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.477368 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.484535 (+ 7167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.484536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.484537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.484546 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.486585 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.487000 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.487001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.487002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.487069 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.487304 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.487305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.487305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.487480 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.487572 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.487743 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.487837 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":69,"threads_started":1}
I20260812 03:52:52.489192 24564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:52.491219 24564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:52.492184 25450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: NON_VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: true } } }
I20260812 03:52:52.492306 25450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:52.493160 26037 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: attempting to promote NON_VOTER 3814bf0897484478a2af880fb7544cdf to VOTER
I20260812 03:52:52.493325 26037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } }
I20260812 03:52:52.493808 25504 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 LEARNER]: Refusing update from remote peer 25d39d64d04b4b71b2e72682026b02a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 03:52:52.493872 25267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Refusing update from remote peer 25d39d64d04b4b71b2e72682026b02a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 03:52:52.494020 25450 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 03:52:52.494119 26037 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 03:52:52.494937 25450 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3814bf0897484478a2af880fb7544cdf (127.17.54.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:52.495131 25504 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3814bf0897484478a2af880fb7544cdf (127.17.54.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:52.495944 25450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:52.496037 25450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:52.496119 25450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:52.496181 25450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:52.496121 25536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:52.496218 25536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:52.496264 25450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:52.496326 25450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:52.496805 25267 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3814bf0897484478a2af880fb7544cdf (127.17.54.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:52.497905 26048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25d39d64d04b4b71b2e72682026b02a1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:52.498031 26048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:52.503162 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.499075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:42359 (local address 127.0.0.1:37944)
0812 03:52:52.499411 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.499422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.499459 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.499976 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.499976 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.499978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:52.500011 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.500012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.500958 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.500959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.501903 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.501907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.502925 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.502926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.502991 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.502996 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.503027 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.503072 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":34,"thread_start_us":113,"threads_started":1}
I20260812 03:52:52.503712 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.499066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46153 (local address 127.0.0.1:43720)
0812 03:52:52.499350 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.499364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.499426 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.499793 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.499795 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.499798 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:52.499867 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.499870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.500992 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.500994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.502361 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.502364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.503466 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.503467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.503536 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.503540 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.503595 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.503637 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260812 03:52:52.504009 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.499233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43720 (local address 127.17.54.193:46153)
0812 03:52:52.499584 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.499591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.499604 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:52:52.499654 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.499657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.499678 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.499779 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:52.500071 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.500880 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.502454 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.503264 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.503343 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.503430 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.503776 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.503912 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":78,"threads_started":1}
I20260812 03:52:52.504205 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.499567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.17.54.194:42359)
0812 03:52:52.499787 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.499792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.499802 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:52.499844 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.499847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.499863 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.499987 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:52.500074 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.500848 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.502001 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.502820 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.503920 (+ 1100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.504004 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.504030 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.504101 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":103,"threads_started":1}
I20260812 03:52:52.504835 25787 tablet_service.cc:1467] Processing CreateTablet for tablet 94451c7c4fad461ab5445d348ed87d25 (DEFAULT_TABLE table=test-table [id=c654447321a74c0bbae31d3eecf88fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:52.504835 25622 tablet_service.cc:1467] Processing CreateTablet for tablet 94451c7c4fad461ab5445d348ed87d25 (DEFAULT_TABLE table=test-table [id=c654447321a74c0bbae31d3eecf88fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:52.505257 25622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94451c7c4fad461ab5445d348ed87d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:52.505257 25787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94451c7c4fad461ab5445d348ed87d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:52.509570 26055 tablet_bootstrap.cc:492] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: Bootstrap starting.
I20260812 03:52:52.509546 26056 tablet_bootstrap.cc:492] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Bootstrap starting.
I20260812 03:52:52.510452 26056 tablet_bootstrap.cc:654] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:52.510452 26055 tablet_bootstrap.cc:654] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:52.510697 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.499847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49036 (local address 127.17.54.195:42841)
0812 03:52:52.501102 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.501107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.501117 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:52.501161 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.501163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.501182 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.501273 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:52.501447 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.502609 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.503752 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.510145 (+ 6393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.510257 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.510425 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.510517 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.510573 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":91,"threads_started":1}
I20260812 03:52:52.510718 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.499172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:42841 (local address 127.0.0.1:49036)
0812 03:52:52.500029 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.500040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.500088 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.501286 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.501287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.501289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:52.501348 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.501350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.502702 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.502704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.503642 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.503645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.510260 (+ 6615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.510262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.510406 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.510413 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.510498 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.510563 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":128,"threads_started":1}
I20260812 03:52:52.510864 26056 log.cc:824] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:52.511600 26055 log.cc:824] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:52.511822 26056 tablet_bootstrap.cc:492] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: No bootstrap required, opened a new log
I20260812 03:52:52.511911 26056 ts_tablet_manager.cc:1397] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:52:52.511758 25943 tablet_service.cc:1467] Processing CreateTablet for tablet 94451c7c4fad461ab5445d348ed87d25 (DEFAULT_TABLE table=test-table [id=c654447321a74c0bbae31d3eecf88fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:52.512173 25943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94451c7c4fad461ab5445d348ed87d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:52.512557 26055 tablet_bootstrap.cc:492] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: No bootstrap required, opened a new log
I20260812 03:52:52.512671 26055 ts_tablet_manager.cc:1397] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:52:52.514904 26056 raft_consensus.cc:348] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.515094 26056 raft_consensus.cc:374] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:52.515141 26056 raft_consensus.cc:729] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b21f85a09e64588806be1c1e5ce1c79, State: Initialized, Role: FOLLOWER
I20260812 03:52:52.515115 26059 tablet_bootstrap.cc:492] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: Bootstrap starting.
I20260812 03:52:52.515266 26056 consensus_queue.cc:260] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.515547 26056 ts_tablet_manager.cc:1428] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 03:52:52.515676 25863 heartbeater.cc:503] Master 127.17.54.254:42117 was elected leader, sending a full tablet report...
I20260812 03:52:52.516201 26055 raft_consensus.cc:348] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.516376 26055 raft_consensus.cc:374] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:52.516424 26055 raft_consensus.cc:729] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c3658deb024df3891ff955776281c4, State: Initialized, Role: FOLLOWER
I20260812 03:52:52.516568 26055 consensus_queue.cc:260] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.516424 26059 tablet_bootstrap.cc:654] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:52.516882 26055 ts_tablet_manager.cc:1428] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 03:52:52.516968 25702 heartbeater.cc:503] Master 127.17.54.254:42117 was elected leader, sending a full tablet report...
I20260812 03:52:52.517045 26059 log.cc:824] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:52.518079 26059 tablet_bootstrap.cc:492] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: No bootstrap required, opened a new log
I20260812 03:52:52.518153 26059 ts_tablet_manager.cc:1397] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:52:52.520366 26059 raft_consensus.cc:348] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.520524 26059 raft_consensus.cc:374] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:52.520547 26059 raft_consensus.cc:729] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b801635b38ab4450b56f8e93d80d5a78, State: Initialized, Role: FOLLOWER
I20260812 03:52:52.520678 26059 consensus_queue.cc:260] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.521041 26059 ts_tablet_manager.cc:1428] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 03:52:52.521116 26017 heartbeater.cc:503] Master 127.17.54.254:42117 was elected leader, sending a full tablet report...
I20260812 03:52:52.543716 26060 raft_consensus.cc:482] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:52.543861 26060 raft_consensus.cc:504] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.544324 26060 leader_election.cc:288] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b801635b38ab4450b56f8e93d80d5a78 (127.17.54.195:42841), b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153)
I20260812 03:52:52.548051 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.544562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:42841 (local address 127.17.54.194:49335)
0812 03:52:52.544637 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.544651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.544700 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.544931 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.544932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.544934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:52.545079 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.545082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.546131 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.546133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.547179 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.547183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.547759 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.547760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.547849 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.547854 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.547910 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.547948 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":25}
I20260812 03:52:52.548066 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.544690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:49335 (local address 127.17.54.195:42841)
0812 03:52:52.544766 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.544769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.544780 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:52.544814 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.544815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.544827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.544910 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:52.545218 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.546030 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.547266 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.547664 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.547731 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.547739 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.547942 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.547994 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:52:52.548152 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.544705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:45115 (local address 127.17.54.193:46153)
0812 03:52:52.544764 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.544767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.544780 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:52:52.544814 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.544815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.544827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.544910 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:52.545223 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.546029 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.547307 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.547733 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.547818 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.547830 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.548018 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.548079 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:52:52.548195 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.544562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46153 (local address 127.17.54.194:45115)
0812 03:52:52.544653 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:52.544662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:52.544700 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:52.544931 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:52.544932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:52.544934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:52.545079 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.545082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.546145 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.546148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.547208 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:52.547211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.547837 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.547837 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:52.547928 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.547932 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:52.547991 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:52:52.548032 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":72}
I20260812 03:52:52.548524 25965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94451c7c4fad461ab5445d348ed87d25" candidate_uuid: "9b21f85a09e64588806be1c1e5ce1c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b801635b38ab4450b56f8e93d80d5a78" is_pre_election: true
I20260812 03:52:52.548517 25644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94451c7c4fad461ab5445d348ed87d25" candidate_uuid: "9b21f85a09e64588806be1c1e5ce1c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c3658deb024df3891ff955776281c4" is_pre_election: true
I20260812 03:52:52.548696 25965 raft_consensus.cc:2454] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b21f85a09e64588806be1c1e5ce1c79 in term 0.
I20260812 03:52:52.548739 25644 raft_consensus.cc:2454] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b21f85a09e64588806be1c1e5ce1c79 in term 0.
I20260812 03:52:52.548940 25733 leader_election.cc:302] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b21f85a09e64588806be1c1e5ce1c79, b801635b38ab4450b56f8e93d80d5a78; no voters:
I20260812 03:52:52.549096 26060 raft_consensus.cc:2781] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:52.549163 26060 raft_consensus.cc:482] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:52.549212 26060 raft_consensus.cc:3037] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:52.550042 26060 raft_consensus.cc:504] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.550208 26060 leader_election.cc:288] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [CANDIDATE]: Term 1 election: Requested vote from peers b801635b38ab4450b56f8e93d80d5a78 (127.17.54.195:42841), b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153)
W20260812 03:52:52.550422 25866 tablet.cc:2369] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:52.550472 25644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94451c7c4fad461ab5445d348ed87d25" candidate_uuid: "9b21f85a09e64588806be1c1e5ce1c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c3658deb024df3891ff955776281c4"
I20260812 03:52:52.550561 25644 raft_consensus.cc:3037] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:52.550675 25965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94451c7c4fad461ab5445d348ed87d25" candidate_uuid: "9b21f85a09e64588806be1c1e5ce1c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b801635b38ab4450b56f8e93d80d5a78"
I20260812 03:52:52.550788 25965 raft_consensus.cc:3037] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:52.551399 25644 raft_consensus.cc:2454] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b21f85a09e64588806be1c1e5ce1c79 in term 1.
I20260812 03:52:52.551592 25732 leader_election.cc:302] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b21f85a09e64588806be1c1e5ce1c79, b2c3658deb024df3891ff955776281c4; no voters:
I20260812 03:52:52.551652 25965 raft_consensus.cc:2454] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b21f85a09e64588806be1c1e5ce1c79 in term 1.
I20260812 03:52:52.551695 26060 raft_consensus.cc:2781] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:52.551843 26060 raft_consensus.cc:686] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 LEADER]: Becoming Leader. State: Replica: 9b21f85a09e64588806be1c1e5ce1c79, State: Running, Role: LEADER
I20260812 03:52:52.551954 26060 consensus_queue.cc:237] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:52.552861 24564 catalog_manager.cc:5495] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b21f85a09e64588806be1c1e5ce1c79 (127.17.54.194). New cstate: current_term: 1 leader_uuid: "9b21f85a09e64588806be1c1e5ce1c79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:52.563179 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.560751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.17.54.194:42359)
0812 03:52:52.560812 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.560814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.560830 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:52:52.560911 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.560912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.560928 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.560987 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:52.561125 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.561610 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.562425 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.562625 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.562680 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.562834 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:52.562939 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:52.562999 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.563070 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.563090 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:52:52.565055 25644 raft_consensus.cc:1261] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 1 FOLLOWER]: Refusing update from remote peer 9b21f85a09e64588806be1c1e5ce1c79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:52.565064 25965 raft_consensus.cc:1261] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 1 FOLLOWER]: Refusing update from remote peer 9b21f85a09e64588806be1c1e5ce1c79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:52.565419 26060 consensus_queue.cc:1035] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:52.565536 26064 consensus_queue.cc:1035] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:52.570283 26070 mvcc.cc:205] Tried to move back new op lower bound from 7317531740423712768 to 7317531740373504000. Current Snapshot: MvccSnapshot[applied={T|T < 7317531740423712768 or (T in {7317531740423712768})}]
I20260812 03:52:52.570461 26071 mvcc.cc:205] Tried to move back new op lower bound from 7317531740423712768 to 7317531740373504000. Current Snapshot: MvccSnapshot[applied={T|T < 7317531740423712768}]
I20260812 03:52:52.570760 24564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:52.570881 24564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:52.571153 26072 mvcc.cc:205] Tried to move back new op lower bound from 7317531740423712768 to 7317531740373504000. Current Snapshot: MvccSnapshot[applied={T|T < 7317531740423712768}]
I20260812 03:52:52.572321 24564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25d39d64d04b4b71b2e72682026b02a1: Sending DeleteTablet for 3 replicas of tablet 94451c7c4fad461ab5445d348ed87d25
I20260812 03:52:52.572690 25787 tablet_service.cc:1514] Processing DeleteTablet for tablet 94451c7c4fad461ab5445d348ed87d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:52 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:37944
I20260812 03:52:52.572695 25943 tablet_service.cc:1514] Processing DeleteTablet for tablet 94451c7c4fad461ab5445d348ed87d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:52 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:49036
I20260812 03:52:52.572976 26084 tablet_replica.cc:330] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: stopping tablet replica
I20260812 03:52:52.573083 26084 raft_consensus.cc:2229] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:52.573128 25622 tablet_service.cc:1514] Processing DeleteTablet for tablet 94451c7c4fad461ab5445d348ed87d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:52 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43720
I20260812 03:52:52.573189 26084 raft_consensus.cc:2258] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:52.573410 26085 tablet_replica.cc:330] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: stopping tablet replica
I20260812 03:52:52.573575 26085 raft_consensus.cc:2229] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:52.573683 26085 raft_consensus.cc:2258] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:52.574031 26083 tablet_replica.cc:330] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: stopping tablet replica
I20260812 03:52:52.574527 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 24531
I20260812 03:52:52.574782 26085 ts_tablet_manager.cc:1905] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:52.578110 26085 ts_tablet_manager.cc:1918] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:52.578377 26085 log.cc:1196] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/94451c7c4fad461ab5445d348ed87d25
I20260812 03:52:52.578791 26084 ts_tablet_manager.cc:1905] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:52.581080 26085 ts_tablet_manager.cc:1939] T 94451c7c4fad461ab5445d348ed87d25 P b2c3658deb024df3891ff955776281c4: Deleting consensus metadata
I20260812 03:52:52.581781 26084 ts_tablet_manager.cc:1918] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:52.581940 26084 log.cc:1196] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/94451c7c4fad461ab5445d348ed87d25
I20260812 03:52:52.582357 26084 ts_tablet_manager.cc:1939] T 94451c7c4fad461ab5445d348ed87d25 P b801635b38ab4450b56f8e93d80d5a78: Deleting consensus metadata
I20260812 03:52:52.584806 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.573614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55960 (local address 127.17.54.252:34973)
0812 03:52:52.573653 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.573655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.573897 (+ 242us) server_negotiation.cc:407] Connection header received
0812 03:52:52.573945 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.573946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.573962 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.574014 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.576816 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.577612 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.579026 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.579346 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.579391 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.581374 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.581376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.581376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.581386 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.583348 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.583796 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.583797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.583798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.583877 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.584122 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.584123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.584123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.584292 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.584412 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.584570 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.584702 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 03:52:52.586061 25890 connection.cc:535] server connection from 127.0.0.1:49036 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 03:52:52.586083 25570 connection.cc:535] server connection from 127.0.0.1:43720 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:52:52.586942 26083 raft_consensus.cc:2229] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:52.587097 26083 raft_consensus.cc:2258] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:52.587661 26083 ts_tablet_manager.cc:1905] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:52.589715 26083 ts_tablet_manager.cc:1918] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:52.589795 26083 log.cc:1196] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/94451c7c4fad461ab5445d348ed87d25
I20260812 03:52:52.590080 26083 ts_tablet_manager.cc:1939] T 94451c7c4fad461ab5445d348ed87d25 P 9b21f85a09e64588806be1c1e5ce1c79: Deleting consensus metadata
W20260812 03:52:52.590566 25732 connection.cc:441] server connection from 127.0.0.1:37944 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37944 (request call id 1) could send its response
I20260812 03:52:52.596760 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.588078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.17.54.253:44003)
0812 03:52:52.588140 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.588142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.588277 (+ 135us) server_negotiation.cc:407] Connection header received
0812 03:52:52.588389 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.588391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.588408 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.588455 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.589419 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.590344 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.591849 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.592162 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.592260 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.593585 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.593586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.593587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.593594 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.595355 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.595879 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.595885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.595885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.595947 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.596152 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.596153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.596154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.596310 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.596423 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.596549 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.596634 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:52:52.597548 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:52.588052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.17.54.252:34973)
0812 03:52:52.588095 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:52.588097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:52.588381 (+ 284us) server_negotiation.cc:407] Connection header received
0812 03:52:52.588476 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:52.588477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:52.588487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:52.588517 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:52.589730 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.590543 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.591318 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:52.591536 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:52.591627 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:52.593535 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:52.593537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:52.593537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:52.593546 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:52.595426 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.595858 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.595859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.595859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.595928 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:52.596467 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:52.596468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:52.596468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:52.596612 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:52.596705 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:52.597334 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 03:52:52.597440 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:52:53.570940 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:53.570541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.193:33003)
0812 03:52:53.570687 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:53.570835 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260812 03:52:53.571051 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:53.570653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.194:43851)
0812 03:52:53.570856 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:53.570930 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":62,"threads_started":1}
W20260812 03:52:53.571251 25863 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:42117 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
I20260812 03:52:53.571506 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:53.571241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.195:60573)
0812 03:52:53.571389 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:53.571465 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":50,"threads_started":1}
W20260812 03:52:53.571557 25702 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:42117 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
W20260812 03:52:53.571740 26017 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:42117 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
I20260812 03:52:54.073351 26099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 25d39d64d04b4b71b2e72682026b02a1)
I20260812 03:52:54.073513 26099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } }
I20260812 03:52:54.073829 26099 leader_election.cc:288] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25d39d64d04b4b71b2e72682026b02a1 (127.17.54.254:42117), 3814bf0897484478a2af880fb7544cdf (127.17.54.252:34973)
I20260812 03:52:54.074252 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.074026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56614)
0812 03:52:54.074144 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.074200 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
W20260812 03:52:54.074535 25235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
I20260812 03:52:54.074542 26102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 25d39d64d04b4b71b2e72682026b02a1)
I20260812 03:52:54.074662 26102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } }
I20260812 03:52:54.075022 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.074907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56630)
0812 03:52:54.074938 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.074968 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260812 03:52:54.075253 25235 leader_election.cc:334] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 25d39d64d04b4b71b2e72682026b02a1 (127.17.54.254:42117): Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
I20260812 03:52:54.075299 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.075104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56634)
0812 03:52:54.075214 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.075262 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
I20260812 03:52:54.075350 26102 leader_election.cc:288] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25d39d64d04b4b71b2e72682026b02a1 (127.17.54.254:42117), 43ce326bae49427c8350369649feafb8 (127.17.54.253:44003)
W20260812 03:52:54.075629 25475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
I20260812 03:52:54.076159 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.076047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56644)
0812 03:52:54.076100 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.076133 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:52:54.076282 25475 leader_election.cc:334] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 25d39d64d04b4b71b2e72682026b02a1 (127.17.54.254:42117): Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
I20260812 03:52:54.077749 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.074415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55970 (local address 127.17.54.252:34973)
0812 03:52:54.074516 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:54.074518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:54.074529 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:54.074562 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:54.074563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:54.074575 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:54.074657 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:54.074895 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.075747 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.076973 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.077377 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.077462 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.077474 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:54.077633 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 03:52:54.077674 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":46,"threads_started":1}
I20260812 03:52:54.077755 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.074221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:34973 (local address 127.0.0.1:55970)
0812 03:52:54.074370 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.074382 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:54.074449 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:54.074722 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:54.074723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:54.074726 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:54.074804 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.074806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.075843 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.075844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.076879 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.076881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.077463 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.077465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.077531 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.077541 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 03:52:54.077625 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:52:54.077672 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":3,"thread_start_us":40,"threads_started":1}
I20260812 03:52:54.078020 25504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43ce326bae49427c8350369649feafb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3814bf0897484478a2af880fb7544cdf" is_pre_election: true
I20260812 03:52:54.078156 25504 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43ce326bae49427c8350369649feafb8 in term 1.
I20260812 03:52:54.078327 25235 leader_election.cc:302] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3814bf0897484478a2af880fb7544cdf, 43ce326bae49427c8350369649feafb8; no voters: 25d39d64d04b4b71b2e72682026b02a1
I20260812 03:52:54.078368 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.075131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:44003 (local address 127.0.0.1:57212)
0812 03:52:54.075359 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.075371 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:54.075445 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:54.075632 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:54.075633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:54.075635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:54.075675 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.075676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.076575 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.076576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.077338 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.077340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.078109 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.078110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.078179 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.078184 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:54.078233 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:52:54.078283 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":57,"threads_started":1}
I20260812 03:52:54.078423 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.075245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.17.54.253:44003)
0812 03:52:54.075391 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:54.075393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:54.075428 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:52:54.075511 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:54.075511 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:54.075521 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:54.075589 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:54.075738 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.076491 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.077435 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.078016 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.078104 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.078115 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:54.078291 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 03:52:54.078353 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260812 03:52:54.078456 26099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:52:54.078496 26099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 25d39d64d04b4b71b2e72682026b02a1)
I20260812 03:52:54.078524 26099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:52:54.078617 25267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3814bf0897484478a2af880fb7544cdf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "43ce326bae49427c8350369649feafb8" is_pre_election: true
I20260812 03:52:54.079309 26099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } }
I20260812 03:52:54.079478 26099 leader_election.cc:288] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [CANDIDATE]: Term 2 election: Requested vote from peers 25d39d64d04b4b71b2e72682026b02a1 (127.17.54.254:42117), 3814bf0897484478a2af880fb7544cdf (127.17.54.252:34973)
I20260812 03:52:54.079504 25267 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3814bf0897484478a2af880fb7544cdf in current term 2: Already voted for candidate 43ce326bae49427c8350369649feafb8 in this term.
I20260812 03:52:54.079648 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.079564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56648)
0812 03:52:54.079594 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.079621 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 03:52:54.079723 25504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43ce326bae49427c8350369649feafb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3814bf0897484478a2af880fb7544cdf"
I20260812 03:52:54.079734 25476 leader_election.cc:302] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3814bf0897484478a2af880fb7544cdf; no voters: 25d39d64d04b4b71b2e72682026b02a1, 43ce326bae49427c8350369649feafb8
I20260812 03:52:54.079830 25504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:52:54.080109 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.079996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56660)
0812 03:52:54.080054 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.080087 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:52:54.080207 25235 leader_election.cc:334] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 25d39d64d04b4b71b2e72682026b02a1 (127.17.54.254:42117): Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
I20260812 03:52:54.080703 25504 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43ce326bae49427c8350369649feafb8 in term 2.
I20260812 03:52:54.080830 26102 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:52:54.080914 25235 leader_election.cc:302] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3814bf0897484478a2af880fb7544cdf, 43ce326bae49427c8350369649feafb8; no voters: 25d39d64d04b4b71b2e72682026b02a1
I20260812 03:52:54.081002 26099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:52:54.081183 26099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [term 2 LEADER]: Becoming Leader. State: Replica: 43ce326bae49427c8350369649feafb8, State: Running, Role: LEADER
I20260812 03:52:54.081274 26099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } }
I20260812 03:52:54.081664 26110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ce326bae49427c8350369649feafb8. Latest consensus state: current_term: 2 leader_uuid: "43ce326bae49427c8350369649feafb8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:54.081792 26110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:54.081984 26112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:54.082707 26112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:54.082814 26112 catalog_manager.cc:1259] Loaded cluster ID: 7b6f610943c94719ac39e64ebffbf72d
I20260812 03:52:54.082849 26112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:54.083025 26112 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:54.083124 26112 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Loaded TSK: 0
I20260812 03:52:54.083264 26112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:54.092275 25254 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:54.093873 25254 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:54.094913 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.094793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56674)
0812 03:52:54.094844 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.094874 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:52:54.094915 25504 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [term 2 FOLLOWER]: Refusing update from remote peer 43ce326bae49427c8350369649feafb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 03:52:54.095147 26099 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 03:52:54.095309 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.095205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.0.0.1:56684)
0812 03:52:54.095257 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.095285 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 03:52:54.095376 25235 consensus_peers.cc:487] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 -> Peer 25d39d64d04b4b71b2e72682026b02a1 (127.17.54.254:42117): Couldn't send request to peer 25d39d64d04b4b71b2e72682026b02a1. Status: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:52:54.095760 26102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ce326bae49427c8350369649feafb8. Latest consensus state: current_term: 2 leader_uuid: "43ce326bae49427c8350369649feafb8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:54.095878 26102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:54.096220 26113 mvcc.cc:205] Tried to move back new op lower bound from 7317531746690617344 to 7317531746637512704. Current Snapshot: MvccSnapshot[applied={T|T < 7317531746690617344}]
I20260812 03:52:54.096316 26110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "43ce326bae49427c8350369649feafb8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:54.096453 26110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:54.096489 26102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "43ce326bae49427c8350369649feafb8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "25d39d64d04b4b71b2e72682026b02a1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42117 } } peers { permanent_uuid: "43ce326bae49427c8350369649feafb8" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 44003 } attrs { promote: false } } peers { permanent_uuid: "3814bf0897484478a2af880fb7544cdf" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 34973 } attrs { promote: false } } }
I20260812 03:52:54.096573 26102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3814bf0897484478a2af880fb7544cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:54.101339 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.098138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:42841 (local address 127.0.0.1:49040)
0812 03:52:54.098170 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.098181 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:54.098224 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:54.098581 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:54.098582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:54.098584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:54.098641 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.098642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.099460 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.099461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.100275 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.100277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.101107 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.101108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.101172 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.101177 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:54.101230 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:52:54.101275 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 03:52:54.101367 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.098254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.17.54.195:42841)
0812 03:52:54.098417 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:54.098419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:54.098426 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:52:54.098469 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:54.098470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:54.098484 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:54.098703 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:54.098745 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.099380 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.100373 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.101028 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.101102 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.101114 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:54.101251 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:52:54.101291 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260812 03:52:54.101627 25943 tablet_service.cc:1467] Processing CreateTablet for tablet 33515904067d45b29e5348b41a9ae4e5 (DEFAULT_TABLE table=test-table [id=1dffcc9ee13c447c8ee6f90dafc4d768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:54.101781 25943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33515904067d45b29e5348b41a9ae4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:54.101953 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.098552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.17.54.193:46153)
0812 03:52:54.098824 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:54.098827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:54.098834 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:52:54.098858 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:54.098858 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:54.098869 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:54.098932 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:54.099075 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.099660 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.100953 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.101580 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.101683 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.101695 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:54.101842 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 03:52:54.101883 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":69,"threads_started":1}
I20260812 03:52:54.101961 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.098456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46153 (local address 127.0.0.1:43726)
0812 03:52:54.098580 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.098590 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:54.098699 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:54.098963 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:54.098965 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:54.098967 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:54.099008 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.099010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.099760 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.099761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.100862 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.100865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.101686 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.101688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.101760 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.101764 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:54.101837 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:52:54.101885 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260812 03:52:54.102201 25622 tablet_service.cc:1467] Processing CreateTablet for tablet 33515904067d45b29e5348b41a9ae4e5 (DEFAULT_TABLE table=test-table [id=1dffcc9ee13c447c8ee6f90dafc4d768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:54.102366 25622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33515904067d45b29e5348b41a9ae4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:54.103223 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.098159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:42359 (local address 127.0.0.1:37960)
0812 03:52:54.098204 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.098212 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:54.098253 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:54.099605 (+ 1352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:54.099606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:54.099608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:54.099650 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.099651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.100520 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.100521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.101554 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.101556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.102986 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.102987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.103064 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.103069 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:54.103100 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 03:52:54.103147 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 03:52:54.103451 26122 tablet_bootstrap.cc:492] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: Bootstrap starting.
I20260812 03:52:54.103665 26123 tablet_bootstrap.cc:492] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4: Bootstrap starting.
I20260812 03:52:54.103724 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.098244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.17.54.194:42359)
0812 03:52:54.099486 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:54.099488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:54.099495 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:52:54.099516 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:54.099517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:54.099526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:54.099602 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:54.099703 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.100422 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.101967 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.102868 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.103552 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.103570 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:54.103589 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 03:52:54.103640 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1222,"thread_start_us":100,"threads_started":1}
I20260812 03:52:54.103864 26122 tablet_bootstrap.cc:654] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:54.103929 25787 tablet_service.cc:1467] Processing CreateTablet for tablet 33515904067d45b29e5348b41a9ae4e5 (DEFAULT_TABLE table=test-table [id=1dffcc9ee13c447c8ee6f90dafc4d768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:54.104058 25787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33515904067d45b29e5348b41a9ae4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:54.104410 26123 tablet_bootstrap.cc:654] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:54.105085 26123 tablet_bootstrap.cc:492] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4: No bootstrap required, opened a new log
I20260812 03:52:54.105139 26123 ts_tablet_manager.cc:1397] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:54.105242 26123 raft_consensus.cc:348] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.105330 26123 raft_consensus.cc:374] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:54.105355 26123 raft_consensus.cc:729] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c3658deb024df3891ff955776281c4, State: Initialized, Role: FOLLOWER
I20260812 03:52:54.105417 26123 consensus_queue.cc:260] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.105589 26123 ts_tablet_manager.cc:1428] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:54.105649 26129 tablet_bootstrap.cc:492] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: Bootstrap starting.
I20260812 03:52:54.106113 26129 tablet_bootstrap.cc:654] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:54.106364 25703 heartbeater.cc:503] Master 127.17.54.253:44003 was elected leader, sending a full tablet report...
I20260812 03:52:54.106842 26129 tablet_bootstrap.cc:492] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: No bootstrap required, opened a new log
I20260812 03:52:54.106897 26129 ts_tablet_manager.cc:1397] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:54.107015 26129 raft_consensus.cc:348] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.107106 26129 raft_consensus.cc:374] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:54.107132 26129 raft_consensus.cc:729] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b21f85a09e64588806be1c1e5ce1c79, State: Initialized, Role: FOLLOWER
I20260812 03:52:54.107112 26122 tablet_bootstrap.cc:492] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: No bootstrap required, opened a new log
I20260812 03:52:54.107184 26122 ts_tablet_manager.cc:1397] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:52:54.107204 26129 consensus_queue.cc:260] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.107308 26122 raft_consensus.cc:348] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.107412 26122 raft_consensus.cc:374] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:54.107491 26122 raft_consensus.cc:729] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b801635b38ab4450b56f8e93d80d5a78, State: Initialized, Role: FOLLOWER
I20260812 03:52:54.107548 26122 consensus_queue.cc:260] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.107740 26122 ts_tablet_manager.cc:1428] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:54.108019 26129 ts_tablet_manager.cc:1428] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:54.108199 26018 heartbeater.cc:503] Master 127.17.54.253:44003 was elected leader, sending a full tablet report...
I20260812 03:52:54.108376 25865 heartbeater.cc:503] Master 127.17.54.253:44003 was elected leader, sending a full tablet report...
I20260812 03:52:54.108609 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.105894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.193:42107)
0812 03:52:54.106442 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.106520 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":115,"threads_started":1}
I20260812 03:52:54.109021 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.108872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.193:42751)
0812 03:52:54.108973 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.109000 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260812 03:52:54.109218 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.107587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.194:50791)
0812 03:52:54.109151 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.109188 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1541,"thread_start_us":109,"threads_started":1}
I20260812 03:52:54.109511 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.108095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.195:40181)
0812 03:52:54.109416 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.109456 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1231,"thread_start_us":92,"threads_started":1}
I20260812 03:52:54.109572 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.109467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.194:37867)
0812 03:52:54.109519 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.109546 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:52:54.132829 25704 tablet.cc:2369] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:54.183408 26134 raft_consensus.cc:482] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:54.183522 26134 raft_consensus.cc:504] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.183935 26134 leader_election.cc:288] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b21f85a09e64588806be1c1e5ce1c79 (127.17.54.194:42359), b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153)
I20260812 03:52:54.187388 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.184176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:42359 (local address 127.17.54.195:34155)
0812 03:52:54.184244 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.184258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:54.184315 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:54.184574 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:54.184576 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:54.184578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:54.184679 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.184682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.185515 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.185516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.186391 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.186394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.187125 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.187126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.187209 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.187214 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:54.187266 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:52:54.187307 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:52:54.187435 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.184235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:34155 (local address 127.17.54.194:42359)
0812 03:52:54.184297 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:54.184300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:54.184314 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:52:54.184450 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:54.184451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:54.184466 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:54.184571 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:54.184765 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.185383 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.186559 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.187037 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.187097 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.187106 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:54.187291 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:52:54.187354 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:52:54.187525 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.184310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:52111 (local address 127.17.54.193:46153)
0812 03:52:54.184355 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:54.184357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:54.184566 (+ 209us) server_negotiation.cc:407] Connection header received
0812 03:52:54.184675 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:54.184677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:54.184687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:54.184764 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:54.184919 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.185542 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.186717 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.187107 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.187182 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.187194 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:54.187401 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 03:52:54.187448 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:54.187727 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.184187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46153 (local address 127.17.54.195:52111)
0812 03:52:54.184491 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.184499 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:54.184594 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:54.184767 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:54.184769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:54.184770 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:54.184841 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.184843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.185650 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.185651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.186632 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:54.186635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.187216 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.187217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:54.187315 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.187320 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:54.187380 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:52:54.187420 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":244,"threads_started":1}
I20260812 03:52:54.187798 25810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33515904067d45b29e5348b41a9ae4e5" candidate_uuid: "b801635b38ab4450b56f8e93d80d5a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b21f85a09e64588806be1c1e5ce1c79" is_pre_election: true
I20260812 03:52:54.187966 25810 raft_consensus.cc:2454] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b801635b38ab4450b56f8e93d80d5a78 in term 0.
I20260812 03:52:54.188060 25644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33515904067d45b29e5348b41a9ae4e5" candidate_uuid: "b801635b38ab4450b56f8e93d80d5a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c3658deb024df3891ff955776281c4" is_pre_election: true
I20260812 03:52:54.188171 25644 raft_consensus.cc:2454] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b801635b38ab4450b56f8e93d80d5a78 in term 0.
I20260812 03:52:54.188179 25892 leader_election.cc:302] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b21f85a09e64588806be1c1e5ce1c79, b801635b38ab4450b56f8e93d80d5a78; no voters:
I20260812 03:52:54.188301 26134 raft_consensus.cc:2781] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:54.188345 26134 raft_consensus.cc:482] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:54.188367 26134 raft_consensus.cc:3037] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:54.189097 26134 raft_consensus.cc:504] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.189252 26134 leader_election.cc:288] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [CANDIDATE]: Term 1 election: Requested vote from peers 9b21f85a09e64588806be1c1e5ce1c79 (127.17.54.194:42359), b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153)
I20260812 03:52:54.189493 25810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33515904067d45b29e5348b41a9ae4e5" candidate_uuid: "b801635b38ab4450b56f8e93d80d5a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b21f85a09e64588806be1c1e5ce1c79"
I20260812 03:52:54.189570 25810 raft_consensus.cc:3037] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:54.189635 25644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33515904067d45b29e5348b41a9ae4e5" candidate_uuid: "b801635b38ab4450b56f8e93d80d5a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c3658deb024df3891ff955776281c4"
I20260812 03:52:54.189710 25644 raft_consensus.cc:3037] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:54.190526 25810 raft_consensus.cc:2454] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b801635b38ab4450b56f8e93d80d5a78 in term 1.
I20260812 03:52:54.190526 25644 raft_consensus.cc:2454] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b801635b38ab4450b56f8e93d80d5a78 in term 1.
I20260812 03:52:54.190711 25892 leader_election.cc:302] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b21f85a09e64588806be1c1e5ce1c79, b801635b38ab4450b56f8e93d80d5a78; no voters:
I20260812 03:52:54.190919 26134 raft_consensus.cc:2781] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:54.191129 26134 raft_consensus.cc:686] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 1 LEADER]: Becoming Leader. State: Replica: b801635b38ab4450b56f8e93d80d5a78, State: Running, Role: LEADER
I20260812 03:52:54.191224 26134 consensus_queue.cc:237] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } }
I20260812 03:52:54.191609 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.191479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.195:47331)
0812 03:52:54.191539 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.191581 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:52:54.192018 25254 catalog_manager.cc:5495] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 reported cstate change: term changed from 0 to 1, leader changed from <none> to b801635b38ab4450b56f8e93d80d5a78 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "b801635b38ab4450b56f8e93d80d5a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b801635b38ab4450b56f8e93d80d5a78" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 } health_report { overall_health: UNKNOWN } } }
W20260812 03:52:54.215555 26019 tablet.cc:2369] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:54.219110 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.216549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49056 (local address 127.17.54.195:42841)
0812 03:52:54.216610 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:54.216613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:54.216621 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:54.216662 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:54.216663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:54.216689 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:54.216766 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:54.216930 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.217433 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.218201 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:54.218463 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:54.218547 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:54.218718 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:54.218846 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:54.218935 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:54.218986 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:52:54.219012 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:52:54.220602 25644 raft_consensus.cc:1261] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 1 FOLLOWER]: Refusing update from remote peer b801635b38ab4450b56f8e93d80d5a78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:54.220642 25810 raft_consensus.cc:1261] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 FOLLOWER]: Refusing update from remote peer b801635b38ab4450b56f8e93d80d5a78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:54.220984 26139 consensus_queue.cc:1035] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b21f85a09e64588806be1c1e5ce1c79" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:54.221131 26134 consensus_queue.cc:1035] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c3658deb024df3891ff955776281c4" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:54.221849 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.221724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.194:51071)
0812 03:52:54.221769 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.221805 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 03:52:54.222139 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.221990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.193:35403)
0812 03:52:54.222069 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.222099 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 03:52:54.223189 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.223032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.195:34943)
0812 03:52:54.223128 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.223155 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 03:52:54.223884 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.223774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.194:60093)
0812 03:52:54.223811 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.223847 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 03:52:54.224596 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.224427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.195:50499)
0812 03:52:54.224533 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.224567 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 03:52:54.225342 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.223167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42117 (local address 127.17.54.193:32925)
0812 03:52:54.225251 (+ 2084us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.225308 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2065}
I20260812 03:52:54.225713 25254 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:54.225816 25254 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:54.227134 25254 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43ce326bae49427c8350369649feafb8: Sending DeleteTablet for 3 replicas of tablet 33515904067d45b29e5348b41a9ae4e5
I20260812 03:52:54.227420 25943 tablet_service.cc:1514] Processing DeleteTablet for tablet 33515904067d45b29e5348b41a9ae4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:54 UTC) from {username='oryx', principal='oryx/127.17.54.253@KRBTEST.COM'} at 127.0.0.1:49040
I20260812 03:52:54.227439 25787 tablet_service.cc:1514] Processing DeleteTablet for tablet 33515904067d45b29e5348b41a9ae4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:54 UTC) from {username='oryx', principal='oryx/127.17.54.253@KRBTEST.COM'} at 127.0.0.1:37960
I20260812 03:52:54.227567 25622 tablet_service.cc:1514] Processing DeleteTablet for tablet 33515904067d45b29e5348b41a9ae4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:54 UTC) from {username='oryx', principal='oryx/127.17.54.253@KRBTEST.COM'} at 127.0.0.1:43726
I20260812 03:52:54.227653 26158 tablet_replica.cc:330] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: stopping tablet replica
I20260812 03:52:54.227716 26158 raft_consensus.cc:2229] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:54.227762 26158 raft_consensus.cc:2258] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:54.227885 26159 tablet_replica.cc:330] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4: stopping tablet replica
I20260812 03:52:54.227953 26159 raft_consensus.cc:2229] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:54.228009 26159 raft_consensus.cc:2258] T 33515904067d45b29e5348b41a9ae4e5 P b2c3658deb024df3891ff955776281c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:54.228147 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 25448
I20260812 03:52:54.235942 26157 tablet_replica.cc:330] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: stopping tablet replica
I20260812 03:52:54.236034 26157 raft_consensus.cc:2229] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:54.236147 26157 raft_consensus.cc:2258] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:54.236392 26158 ts_tablet_manager.cc:1905] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:54.238204 26157 ts_tablet_manager.cc:1905] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:54.240020 26158 ts_tablet_manager.cc:1918] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:54.240098 26158 log.cc:1196] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/33515904067d45b29e5348b41a9ae4e5
I20260812 03:52:54.240437 26158 ts_tablet_manager.cc:1939] T 33515904067d45b29e5348b41a9ae4e5 P 9b21f85a09e64588806be1c1e5ce1c79: Deleting consensus metadata
I20260812 03:52:54.240613 26157 ts_tablet_manager.cc:1918] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:54.240674 26157 log.cc:1196] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/33515904067d45b29e5348b41a9ae4e5
I20260812 03:52:54.240916 26157 ts_tablet_manager.cc:1939] T 33515904067d45b29e5348b41a9ae4e5 P b801635b38ab4450b56f8e93d80d5a78: Deleting consensus metadata
I20260812 03:52:54.240895 25239 catalog_manager.cc:4841] TS 9b21f85a09e64588806be1c1e5ce1c79 (127.17.54.194:42359): tablet 33515904067d45b29e5348b41a9ae4e5 (table test-table [id=1dffcc9ee13c447c8ee6f90dafc4d768]) successfully deleted
I20260812 03:52:54.241457 25236 catalog_manager.cc:4841] TS b801635b38ab4450b56f8e93d80d5a78 (127.17.54.195:42841): tablet 33515904067d45b29e5348b41a9ae4e5 (table test-table [id=1dffcc9ee13c447c8ee6f90dafc4d768]) successfully deleted
I20260812 03:52:54.245534 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:54.245377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46153 (local address 127.0.0.1:43730)
0812 03:52:54.245453 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:54.245489 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:52:54.245720 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 25712
W20260812 03:52:54.245705 25235 catalog_manager.cc:4584] TS b2c3658deb024df3891ff955776281c4 (127.17.54.193:46153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33515904067d45b29e5348b41a9ae4e5: Network error: Client connection negotiation failed: client connection to 127.17.54.193:46153: connect: Connection refused (error 111)
I20260812 03:52:54.254164 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 25870
I20260812 03:52:54.262521 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 25098
I20260812 03:52:54.270105 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 25321
2026-08-12T03:52:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4872 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:54 dist-test-slave-tm1g krb5kdc[26162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:54 dist-test-slave-tm1g krb5kdc[26162](info): set up 2 sockets
Aug 12 03:52:54 dist-test-slave-tm1g krb5kdc[26162](info): commencing operation
krb5kdc: starting...
W20260812 03:52:55.388746 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:55 dist-test-slave-tm1g krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.254.keytab.
Entry for principal oryx/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.254.keytab.
I20260812 03:52:55.517839 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:41863
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.254.keytab
--principal=oryx/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:41863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:55.601274 26176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:55.601547 26176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:55.601588 26176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:55.602806 26176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:55.602881 26176 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:52:55.602905 26176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:55.602926 26176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:55.602947 26176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:55.602977 26176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:55.604562 26176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:41863
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.254.keytab
--principal=oryx/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:41863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:55.604861 26176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:55.605110 26176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:55.610471 26181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:55.610463 26184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:55.610462 26176 server_base.cc:1034] running on GCE node
W20260812 03:52:55.610493 26182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:55.611008 26176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:55.611333 26176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:55.612517 26176 hybrid_clock.cc:648] HybridClock initialized: now 1786506775612494 us; error 38 us; skew 500 ppm
Aug 12 03:52:55 dist-test-slave-tm1g krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506775, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:55.615187 26176 init.cc:377] Logged in from keytab as oryx/127.17.54.254@KRBTEST.COM (short username oryx)
I20260812 03:52:55.615681 26176 webserver.cc:460] Webserver started at http://127.17.54.254:39031/ using document root <none> and password file <none>
I20260812 03:52:55.615924 26176 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:55.615971 26176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:55.616145 26176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:55.617295 26176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "19c0cfad41524fe18335d7eabfd47260"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "eedbac26b08bf30579bf2496425d4131"
server_key_iv: "cfac8c4d3f3b6aeef6b7fb7ca9cf2e03"
server_key_version: "encryptionkey@0"
I20260812 03:52:55.617681 26176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "19c0cfad41524fe18335d7eabfd47260"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "eedbac26b08bf30579bf2496425d4131"
server_key_iv: "cfac8c4d3f3b6aeef6b7fb7ca9cf2e03"
server_key_version: "encryptionkey@0"
I20260812 03:52:55.619197 26176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:55.620085 26193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:55.620281 26176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:55.620433 26176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "19c0cfad41524fe18335d7eabfd47260"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "eedbac26b08bf30579bf2496425d4131"
server_key_iv: "cfac8c4d3f3b6aeef6b7fb7ca9cf2e03"
server_key_version: "encryptionkey@0"
I20260812 03:52:55.620514 26176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:55.635527 26176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:55.637930 26176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:55.638101 26176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:55.643508 26176 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:41863
I20260812 03:52:55.643580 26267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:41863 every 8 connection(s)
I20260812 03:52:55.643957 26176 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:55.644708 26268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:55.647320 26268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: Bootstrap starting.
I20260812 03:52:55.647980 26268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:55.648308 26268 log.cc:824] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:55.649060 26268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: No bootstrap required, opened a new log
I20260812 03:52:55.651288 26268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c0cfad41524fe18335d7eabfd47260" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41863 } }
I20260812 03:52:55.651436 26268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:55.651459 26268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c0cfad41524fe18335d7eabfd47260, State: Initialized, Role: FOLLOWER
I20260812 03:52:55.651623 26268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c0cfad41524fe18335d7eabfd47260" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41863 } }
I20260812 03:52:55.651702 26268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:55.651753 26268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:55.651804 26268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:55.652377 26268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c0cfad41524fe18335d7eabfd47260" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41863 } }
I20260812 03:52:55.652470 26268 leader_election.cc:302] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19c0cfad41524fe18335d7eabfd47260; no voters:
I20260812 03:52:55.652679 26268 leader_election.cc:288] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:55.652757 26271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:55.652951 26271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [term 1 LEADER]: Becoming Leader. State: Replica: 19c0cfad41524fe18335d7eabfd47260, State: Running, Role: LEADER
I20260812 03:52:55.652983 26268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:55.653049 26271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c0cfad41524fe18335d7eabfd47260" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41863 } }
I20260812 03:52:55.653501 26272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19c0cfad41524fe18335d7eabfd47260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c0cfad41524fe18335d7eabfd47260" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41863 } } }
I20260812 03:52:55.653510 26273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c0cfad41524fe18335d7eabfd47260. Latest consensus state: current_term: 1 leader_uuid: "19c0cfad41524fe18335d7eabfd47260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c0cfad41524fe18335d7eabfd47260" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 41863 } } }
I20260812 03:52:55.653638 26272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:55.653864 26280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:55.654000 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 26176
I20260812 03:52:55.654142 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:55.654399 17627 external_mini_cluster.cc:1428] Setting key c4f1860c9aa1d92f53950ebc68776b1b
I20260812 03:52:55.654518 26280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:55.653647 26273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:55.656225 26280 catalog_manager.cc:1347] Generated new cluster ID: b3e4570ec0ee4fda9ff9d5a4c7dd3851
I20260812 03:52:55.656281 26280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:52:55 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:55.666854 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:55.655053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.17.54.254:41863)
0812 03:52:55.655253 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:55.655263 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:55.655295 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:52:55.655424 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:55.655432 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:55.655492 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:55.655643 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:55.656543 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.657897 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.659046 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.659701 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.659801 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:55.662844 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:55.662849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:55.662849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:55.662863 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:55.664994 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:55.665422 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:55.665426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:55.665427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:55.665566 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:55.665781 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:55.665782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:55.665782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:55.666127 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:55.666338 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:55.666444 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 03:52:55.666615 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260812 03:52:55.667668 26280 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:55.668159 26280 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:55.671697 26280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: Generated new TSK 0
I20260812 03:52:55.671886 26280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab.
Entry for principal oryx/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab.
I20260812 03:52:55.705302 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41863
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab
--principal=oryx/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:55.820161 26295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:55.820386 26295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:55.820420 26295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:55.821719 26295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:55.821761 26295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:55.821784 26295 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:55.821801 26295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:55.823606 26295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab
--principal=oryx/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:55.823877 26295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:55.824137 26295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:55.829706 26304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:55.829674 26301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:55.829682 26302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:55.830104 26295 server_base.cc:1034] running on GCE node
I20260812 03:52:55.830358 26295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:55.830636 26295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:55.831804 26295 hybrid_clock.cc:648] HybridClock initialized: now 1786506775831786 us; error 33 us; skew 500 ppm
Aug 12 03:52:55 dist-test-slave-tm1g krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506775, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:55.834487 26295 init.cc:377] Logged in from keytab as oryx/127.17.54.193@KRBTEST.COM (short username oryx)
I20260812 03:52:55.834961 26295 webserver.cc:460] Webserver started at http://127.17.54.193:37765/ using document root <none> and password file <none>
I20260812 03:52:55.835227 26295 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:55.835299 26295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:55.835464 26295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:55.836570 26295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "4c97b68889234966ba5fd2291e647d4d"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "d9058b7da5ae6ffc2eff75e9fdded14b"
server_key_iv: "891b2fb557190b11d5cb538e9fb6ccf6"
server_key_version: "encryptionkey@0"
I20260812 03:52:55.836941 26295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "4c97b68889234966ba5fd2291e647d4d"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "d9058b7da5ae6ffc2eff75e9fdded14b"
server_key_iv: "891b2fb557190b11d5cb538e9fb6ccf6"
server_key_version: "encryptionkey@0"
I20260812 03:52:55.838370 26295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:55.839180 26313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:55.839360 26295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:55.839463 26295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "4c97b68889234966ba5fd2291e647d4d"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "d9058b7da5ae6ffc2eff75e9fdded14b"
server_key_iv: "891b2fb557190b11d5cb538e9fb6ccf6"
server_key_version: "encryptionkey@0"
I20260812 03:52:55.839571 26295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:55.849781 26295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:55.851743 26295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:55.851944 26295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:55.852490 26295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:55.852558 26295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:55.852605 26295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:55.852635 26295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:52:55 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506775, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:55.860517 26295 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:38971
I20260812 03:52:55.860877 26295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:55.860600 26443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:38971 every 8 connection(s)
I20260812 03:52:55.862195 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 26295
I20260812 03:52:55.862284 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:55.862548 17627 external_mini_cluster.cc:1428] Setting key f32fa1578f8445d604d55fc3d7f4fb61
I20260812 03:52:55.863111 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:55.853202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:46541 (local address 127.17.54.254:41863)
0812 03:52:55.853275 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:55.853278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:55.853499 (+ 221us) server_negotiation.cc:407] Connection header received
0812 03:52:55.854197 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:55.854199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:55.854217 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:55.854268 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:55.855000 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.855563 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.856637 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.856853 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.856946 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:55.859796 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:55.859798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:55.859799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:55.859807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:55.861536 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:55.861970 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:55.861972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:55.861972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:55.862040 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:55.862342 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:55.862343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:55.862344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:55.862495 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:55.862575 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:55.862922 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 03:52:55.863018 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:52:55.863298 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:55.853153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41863 (local address 127.17.54.193:46541)
0812 03:52:55.853345 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:55.853382 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:55.854075 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:55.854375 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:55.854380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:55.854694 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:55.854893 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:55.854897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.855667 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.855668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:55.856543 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:55.856545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.856962 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.856963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:55.857153 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:55.857772 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:55.857783 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:55.859687 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:55.861632 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:55.861638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:55.861640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:55.861871 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:55.862110 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:55.862112 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:55.862112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:55.862229 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:55.862581 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:55.862589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:55.862772 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 03:52:55.862882 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 03:52:55.863062 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260812 03:52:55.863808 26444 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41863
I20260812 03:52:55.863921 26444 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:55.864109 26444 heartbeater.cc:511] Master 127.17.54.254:41863 requested a full tablet report, sending...
I20260812 03:52:55.864675 26214 ts_manager.cc:194] Registered new tserver with Master: 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971)
I20260812 03:52:55.865358 26214 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:46541
I20260812 03:52:55.871582 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:55.865584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:45147 (local address 127.17.54.254:41863)
0812 03:52:55.865633 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:55.865635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:55.865843 (+ 208us) server_negotiation.cc:407] Connection header received
0812 03:52:55.865906 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:55.865907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:55.865917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:55.865973 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:55.866366 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.866849 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.867570 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.867840 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.867906 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:55.868905 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:55.868907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:55.868907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:55.868915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:55.870378 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:55.870746 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:55.870747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:55.870748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:55.870806 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:55.871023 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:55.871024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:55.871025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:55.871148 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:55.871218 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:55.871388 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 03:52:55.871475 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:52:55.871591 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:55.865637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41863 (local address 127.17.54.193:45147)
0812 03:52:55.865783 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:55.865796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:55.865857 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:55.865978 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:55.865979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:55.866177 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:55.866242 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:55.866244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.866958 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.866959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:55.867483 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:55.867484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:55.867904 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:55.867906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:55.867978 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:55.868495 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:55.868499 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:55.868801 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:55.870478 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:55.870479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:55.870480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:55.870669 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:55.870884 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:55.870885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:55.870886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:55.870954 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:55.871222 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:55.871223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:55.871272 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 03:52:55.871350 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:52:55.871476 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab.
Entry for principal oryx/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab.
I20260812 03:52:55.893653 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41863
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab
--principal=oryx/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:55.985464 26455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:55.985733 26455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:55.985780 26455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:55.987128 26455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:55.987201 26455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:55.987241 26455 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:55.987299 26455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:52:55.988984 26455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab
--principal=oryx/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:55.989285 26455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:55.989542 26455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:55.994880 26464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:55.994881 26460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:55.994900 26461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:55.995288 26455 server_base.cc:1034] running on GCE node
I20260812 03:52:55.995503 26455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:55.995750 26455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:55.996909 26455 hybrid_clock.cc:648] HybridClock initialized: now 1786506775996863 us; error 62 us; skew 500 ppm
Aug 12 03:52:55 dist-test-slave-tm1g krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506775, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:55.999682 26455 init.cc:377] Logged in from keytab as oryx/127.17.54.194@KRBTEST.COM (short username oryx)
I20260812 03:52:56.000208 26455 webserver.cc:460] Webserver started at http://127.17.54.194:33443/ using document root <none> and password file <none>
I20260812 03:52:56.000479 26455 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:56.000552 26455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:56.000708 26455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:56.001824 26455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "fe5d67f11bd3404aab41649646e08a1e"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "125d35d624ddb0f65be5210154165517"
server_key_iv: "6a25e6c6c8e537b553a76f47180202a7"
server_key_version: "encryptionkey@0"
I20260812 03:52:56.002214 26455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "fe5d67f11bd3404aab41649646e08a1e"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "125d35d624ddb0f65be5210154165517"
server_key_iv: "6a25e6c6c8e537b553a76f47180202a7"
server_key_version: "encryptionkey@0"
I20260812 03:52:56.003809 26455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:56.004969 26473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.005235 26455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:56.005381 26455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "fe5d67f11bd3404aab41649646e08a1e"
format_stamp: "Formatted at 2026-08-12 03:52:55 on dist-test-slave-tm1g"
server_key: "125d35d624ddb0f65be5210154165517"
server_key_iv: "6a25e6c6c8e537b553a76f47180202a7"
server_key_version: "encryptionkey@0"
I20260812 03:52:56.005457 26455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:56.021183 26455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:56.024068 26455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:56.024271 26455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:56.024847 26455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:56.024917 26455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.024968 26455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:56.024996 26455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.032164 26455 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:43173
I20260812 03:52:56.032217 26604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:43173 every 8 connection(s)
I20260812 03:52:56.032577 26455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506775, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:56.036252 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.025484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:35625 (local address 127.17.54.254:41863)
0812 03:52:56.025538 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.025540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.025848 (+ 308us) server_negotiation.cc:407] Connection header received
0812 03:52:56.026475 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.026477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.026495 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.026593 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.027495 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.028357 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.030040 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.030407 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.030512 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.033321 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:56.033324 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:56.033324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:56.033332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:56.034998 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.035426 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.035427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.035427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.035480 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.035681 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.035681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.035682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.035791 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:56.035838 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.036104 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.036182 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:52:56.036419 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.025480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41863 (local address 127.17.54.194:35625)
0812 03:52:56.025652 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.025737 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.026290 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.026646 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.026650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.027070 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:56.027384 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.027392 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.028472 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.028474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.029950 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.029956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.030524 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.030525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.030678 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.031314 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:56.031324 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:56.033172 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:56.035099 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.035107 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.035109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.035357 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.035537 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.035537 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.035538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.035641 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.035840 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:56.035844 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:56.035975 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.036068 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.036204 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260812 03:52:56.037048 26605 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41863
I20260812 03:52:56.037151 26605 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:56.037362 26605 heartbeater.cc:511] Master 127.17.54.254:41863 requested a full tablet report, sending...
I20260812 03:52:56.037878 26215 ts_manager.cc:194] Registered new tserver with Master: fe5d67f11bd3404aab41649646e08a1e (127.17.54.194:43173)
I20260812 03:52:56.038468 26215 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:35625
I20260812 03:52:56.040207 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 26455
I20260812 03:52:56.040372 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:52:56.040594 17627 external_mini_cluster.cc:1428] Setting key 38771ffc0ef79adc71cf0b2b7e3c7f3d
I20260812 03:52:56.044445 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.038328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:38675 (local address 127.17.54.254:41863)
0812 03:52:56.038376 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.038378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.038448 (+ 70us) server_negotiation.cc:407] Connection header received
0812 03:52:56.038493 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.038494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.038503 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.038595 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.038948 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.039435 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.040180 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.040483 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.040563 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.041610 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:56.041612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:56.041612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:56.041620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:56.043099 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.043570 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.043571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.043572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.043636 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.043845 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.043846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.043846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.044018 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:56.044097 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.044257 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.044345 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:52:56.044461 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.038160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41863 (local address 127.17.54.194:38675)
0812 03:52:56.038375 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.038385 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.038443 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.038601 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.038602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.038803 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:56.038865 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.038867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.039536 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.039537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.040089 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.040090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.040561 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.040562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.040641 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.041132 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:56.041137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:56.041521 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:56.043189 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.043190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.043191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.043475 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.043708 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.043709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.043709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.043777 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.044087 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:56.044088 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:56.044142 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.044231 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.044347 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for oryx/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab.
Entry for principal oryx/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab.
I20260812 03:52:56.070816 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:41863
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab
--principal=oryx/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:56.154222 26617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:56.154491 26617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:56.154520 26617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:56.155885 26617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:56.155941 26617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:56.155963 26617 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:52:56.155982 26617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:52:56.157591 26617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab
--principal=oryx/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.54.254:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:56.157846 26617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:56.158082 26617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:56.163233 26627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:56.163260 26625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:56.163275 26624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:56.163568 26617 server_base.cc:1034] running on GCE node
I20260812 03:52:56.163816 26617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:56.164147 26617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:56.165305 26617 hybrid_clock.cc:648] HybridClock initialized: now 1786506776165296 us; error 28 us; skew 500 ppm
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506776, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:56.167994 26617 init.cc:377] Logged in from keytab as oryx/127.17.54.195@KRBTEST.COM (short username oryx)
I20260812 03:52:56.168408 26617 webserver.cc:460] Webserver started at http://127.17.54.195:38241/ using document root <none> and password file <none>
I20260812 03:52:56.168629 26617 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:56.168671 26617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:56.168766 26617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:56.169790 26617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "ab02d3d36ff34c3f848c500563938e13"
format_stamp: "Formatted at 2026-08-12 03:52:56 on dist-test-slave-tm1g"
server_key: "19fcfc3decb9d871ecbd4b44ca0fa2e6"
server_key_iv: "fd63e81ee1b72282d0cc807ce173b069"
server_key_version: "encryptionkey@0"
I20260812 03:52:56.170109 26617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "ab02d3d36ff34c3f848c500563938e13"
format_stamp: "Formatted at 2026-08-12 03:52:56 on dist-test-slave-tm1g"
server_key: "19fcfc3decb9d871ecbd4b44ca0fa2e6"
server_key_iv: "fd63e81ee1b72282d0cc807ce173b069"
server_key_version: "encryptionkey@0"
I20260812 03:52:56.171561 26617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:56.172451 26636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.172619 26617 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:56.172724 26617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "ab02d3d36ff34c3f848c500563938e13"
format_stamp: "Formatted at 2026-08-12 03:52:56 on dist-test-slave-tm1g"
server_key: "19fcfc3decb9d871ecbd4b44ca0fa2e6"
server_key_iv: "fd63e81ee1b72282d0cc807ce173b069"
server_key_version: "encryptionkey@0"
I20260812 03:52:56.172801 26617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:56.209470 26617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:56.211921 26617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:56.212044 26617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:56.212544 26617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:56.212581 26617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.212610 26617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:56.212663 26617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.220077 26617 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:35263
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506776, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:56.220149 26763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:35263 every 8 connection(s)
I20260812 03:52:56.220538 26617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:52:56.223882 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.213172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:60133 (local address 127.17.54.254:41863)
0812 03:52:56.213221 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.213227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.213475 (+ 248us) server_negotiation.cc:407] Connection header received
0812 03:52:56.214157 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.214159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.214176 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.214246 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.214938 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.215801 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.217502 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.217853 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.217945 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.220725 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:56.220726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:56.220727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:56.220750 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:56.222338 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.222867 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.222868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.222868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.222926 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.223170 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.223171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.223171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.223326 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:56.223393 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.223742 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.223815 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:52:56.224174 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.213178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41863 (local address 127.17.54.195:60133)
0812 03:52:56.213350 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.213388 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.213998 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.214381 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.214385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.214647 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:56.214849 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.214854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.215907 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.215909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.217389 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.217394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.217961 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.217962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.218077 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.218745 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:56.218753 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:56.220623 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:56.222441 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.222449 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.222451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.222777 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.222991 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.222992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.222992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.223107 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.223421 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:56.223428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:56.223590 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.223717 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.223909 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260812 03:52:56.224731 26764 heartbeater.cc:347] Connected to a master server at 127.17.54.254:41863
I20260812 03:52:56.224844 26764 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:56.225070 26764 heartbeater.cc:511] Master 127.17.54.254:41863 requested a full tablet report, sending...
I20260812 03:52:56.225628 26215 ts_manager.cc:194] Registered new tserver with Master: ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.226191 26215 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:60133
I20260812 03:52:56.227131 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 26617
I20260812 03:52:56.227224 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:52:56.227440 17627 external_mini_cluster.cc:1428] Setting key 33d6d617c693f25bc697616ee02588cc
I20260812 03:52:56.228394 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:52:56.232843 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.225868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41863 (local address 127.17.54.195:38547)
0812 03:52:56.225998 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.226007 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.226063 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.226208 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.226209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.226436 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:56.226501 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.226503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.227326 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.227327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.228156 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.228158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.228722 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.228723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.228799 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.229300 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:56.229306 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:56.229749 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:56.231767 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.231767 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.231768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.231946 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.232179 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.232180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.232181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.232249 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.232564 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:56.232565 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:56.232610 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.232670 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.232764 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260812 03:52:56.232937 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.225936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:38547 (local address 127.17.54.254:41863)
0812 03:52:56.225986 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.225988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.226069 (+ 81us) server_negotiation.cc:407] Connection header received
0812 03:52:56.226117 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.226117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.226129 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.226193 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.226616 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.227201 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.228404 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.228649 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.228708 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.229842 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:56.229843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:56.229843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:56.229850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:56.231669 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.232023 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.232025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.232025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.232088 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.232325 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.232326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.232327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.232490 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:56.232560 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.232717 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.232806 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506776, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:52:56.253320 26291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:56.247564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52848 (local address 127.17.54.254:41863)
0812 03:52:56.247635 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.247637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.247646 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:56.247672 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.247673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.247682 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.247733 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.248355 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.248867 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.249670 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.249851 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.249903 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.253220 (+ 3317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52848: BlockingRecv error: recv got EOF from 127.0.0.1:52848 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506776, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:56.261298 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.254273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.17.54.254:41863)
0812 03:52:56.254378 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.254380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.254389 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:56.254446 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.254447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.254455 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.254510 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.255033 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.255526 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.256234 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.256450 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.256505 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.258720 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:56.258722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:56.258722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:56.258730 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:56.260146 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.260511 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.260512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.260513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.260573 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.260774 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.260775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.260776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.260891 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:56.261000 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.261112 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.261190 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 03:52:56.262897 26215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:56.264587 26215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:56.271445 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.267229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:38971 (local address 127.0.0.1:49968)
0812 03:52:56.267639 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.267649 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.267694 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.267960 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.267960 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.267962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:56.268020 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.268021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.268927 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.268928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.270102 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.270105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.271118 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.271119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.271198 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.271211 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.271288 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.271346 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":69,"threads_started":1}
I20260812 03:52:56.271512 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.267396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.17.54.193:38971)
0812 03:52:56.267608 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.267612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.267753 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:52:56.267856 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.267859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.267877 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.267951 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:56.268109 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.268833 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.270195 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.271021 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.271099 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.271174 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.271344 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.271390 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":110,"threads_started":1}
I20260812 03:52:56.271775 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.267168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.17.54.194:43173)
0812 03:52:56.267365 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.267372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.267646 (+ 274us) server_negotiation.cc:407] Connection header received
0812 03:52:56.267776 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.267779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.267795 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.267850 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:56.268121 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.269287 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.270557 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.271306 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.271353 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.271418 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.271653 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.271694 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
I20260812 03:52:56.271878 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.267029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:35263 (local address 127.0.0.1:43450)
0812 03:52:56.267372 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.267414 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.267486 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.267883 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.267886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.267888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:56.267963 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.267966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.269379 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.269380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.270464 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.270466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.271280 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.271281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.271345 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.271347 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.271394 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.271426 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":52,"thread_start_us":80,"threads_started":1}
I20260812 03:52:56.272032 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.267029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43173 (local address 127.0.0.1:33434)
0812 03:52:56.267563 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.267575 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.267679 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.267946 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.267947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.267949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:56.268008 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.268010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.269397 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.269398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.270453 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.270458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.271484 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.271485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.271554 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.271558 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.271916 (+ 358us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.271965 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260812 03:52:56.272233 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.267153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43450 (local address 127.17.54.195:35263)
0812 03:52:56.267442 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.267446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.267457 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:56.267697 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.267701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.267721 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.267798 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:56.268145 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.269273 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.270545 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.271090 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.271162 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.271246 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.271491 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.272121 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":100,"threads_started":1}
I20260812 03:52:56.272367 26370 tablet_service.cc:1467] Processing CreateTablet for tablet 51deb01f19b84dae8bb4af7e7eb340e9 (DEFAULT_TABLE table=test-table [id=f13bcc0bf41648a184be1dca4f2f256a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.272760 26370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51deb01f19b84dae8bb4af7e7eb340e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.272648 26529 tablet_service.cc:1467] Processing CreateTablet for tablet 51deb01f19b84dae8bb4af7e7eb340e9 (DEFAULT_TABLE table=test-table [id=f13bcc0bf41648a184be1dca4f2f256a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.272879 26687 tablet_service.cc:1467] Processing CreateTablet for tablet 51deb01f19b84dae8bb4af7e7eb340e9 (DEFAULT_TABLE table=test-table [id=f13bcc0bf41648a184be1dca4f2f256a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.272975 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51deb01f19b84dae8bb4af7e7eb340e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.273154 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51deb01f19b84dae8bb4af7e7eb340e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.276535 26795 tablet_bootstrap.cc:492] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: Bootstrap starting.
I20260812 03:52:56.277046 26796 tablet_bootstrap.cc:492] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Bootstrap starting.
I20260812 03:52:56.277271 26797 tablet_bootstrap.cc:492] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: Bootstrap starting.
I20260812 03:52:56.277325 26795 tablet_bootstrap.cc:654] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.277645 26796 tablet_bootstrap.cc:654] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.277683 26795 log.cc:824] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:56.277922 26797 tablet_bootstrap.cc:654] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.278026 26796 log.cc:824] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:56.278360 26797 log.cc:824] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:56.278663 26795 tablet_bootstrap.cc:492] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: No bootstrap required, opened a new log
I20260812 03:52:56.278723 26795 ts_tablet_manager.cc:1397] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:56.278963 26796 tablet_bootstrap.cc:492] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: No bootstrap required, opened a new log
I20260812 03:52:56.279044 26796 ts_tablet_manager.cc:1397] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:56.280268 26795 raft_consensus.cc:348] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.280407 26795 raft_consensus.cc:374] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.280429 26795 raft_consensus.cc:729] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab02d3d36ff34c3f848c500563938e13, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.280561 26795 consensus_queue.cc:260] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.280507 26796 raft_consensus.cc:348] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.280684 26796 raft_consensus.cc:374] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.280728 26796 raft_consensus.cc:729] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.280858 26795 ts_tablet_manager.cc:1428] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:52:56.280853 26796 consensus_queue.cc:260] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.280921 26764 heartbeater.cc:503] Master 127.17.54.254:41863 was elected leader, sending a full tablet report...
I20260812 03:52:56.281160 26796 ts_tablet_manager.cc:1428] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:52:56.281288 26605 heartbeater.cc:503] Master 127.17.54.254:41863 was elected leader, sending a full tablet report...
I20260812 03:52:56.282197 26797 tablet_bootstrap.cc:492] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: No bootstrap required, opened a new log
I20260812 03:52:56.282264 26797 ts_tablet_manager.cc:1397] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 03:52:56.282434 26803 raft_consensus.cc:482] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:56.282510 26803 raft_consensus.cc:504] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.282822 26803 leader_election.cc:288] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263), 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971)
W20260812 03:52:56.283196 26607 tablet.cc:2369] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:56.284734 26797 raft_consensus.cc:348] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.284873 26797 raft_consensus.cc:374] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.284895 26797 raft_consensus.cc:729] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c97b68889234966ba5fd2291e647d4d, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.285013 26797 consensus_queue.cc:260] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.285329 26444 heartbeater.cc:503] Master 127.17.54.254:41863 was elected leader, sending a full tablet report...
I20260812 03:52:56.285567 26797 ts_tablet_manager.cc:1428] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 03:52:56.286667 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.282992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:35263 (local address 127.17.54.194:52085)
0812 03:52:56.283073 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.283086 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.283142 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.283486 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.283487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.283489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:56.283613 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.283615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.284518 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.284519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.285678 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.285682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.286418 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.286419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.286527 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.286531 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.286580 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.286618 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 03:52:56.286727 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.283051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:52085 (local address 127.17.54.195:35263)
0812 03:52:56.283231 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.283234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.283245 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:52:56.283271 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.283272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.283282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.283331 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:56.283697 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.284426 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.285757 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.286345 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.286417 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.286425 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.286603 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.286641 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260812 03:52:56.287034 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.283396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:52665 (local address 127.17.54.193:38971)
0812 03:52:56.283438 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.283440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.283448 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:56.283496 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.283496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.283505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.283539 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:56.283744 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.284216 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.286079 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.286656 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.286727 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.286736 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.286909 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.286949 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:52:56.287045 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.283106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:38971 (local address 127.17.54.194:52665)
0812 03:52:56.283226 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.283238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.283429 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.283613 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.283614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.283615 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:56.283691 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.283693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.284309 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.284310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.285888 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.285891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.286742 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.286743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.286826 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.286831 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.286895 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.286938 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":34,"thread_start_us":77,"threads_started":1}
I20260812 03:52:56.287163 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51deb01f19b84dae8bb4af7e7eb340e9" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13" is_pre_election: true
I20260812 03:52:56.287295 26711 raft_consensus.cc:2454] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 0.
I20260812 03:52:56.287323 26391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51deb01f19b84dae8bb4af7e7eb340e9" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c97b68889234966ba5fd2291e647d4d" is_pre_election: true
I20260812 03:52:56.287448 26391 raft_consensus.cc:2454] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 0.
I20260812 03:52:56.287518 26476 leader_election.cc:302] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab02d3d36ff34c3f848c500563938e13, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.287645 26803 raft_consensus.cc:2781] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:56.287699 26803 raft_consensus.cc:482] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:56.287721 26803 raft_consensus.cc:3037] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.288733 26803 raft_consensus.cc:504] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.288877 26803 leader_election.cc:288] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 election: Requested vote from peers ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263), 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971)
I20260812 03:52:56.289008 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51deb01f19b84dae8bb4af7e7eb340e9" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13"
I20260812 03:52:56.289031 26391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51deb01f19b84dae8bb4af7e7eb340e9" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c97b68889234966ba5fd2291e647d4d"
I20260812 03:52:56.289089 26391 raft_consensus.cc:3037] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.289088 26711 raft_consensus.cc:3037] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.289913 26711 raft_consensus.cc:2454] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 1.
I20260812 03:52:56.289913 26391 raft_consensus.cc:2454] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 1.
I20260812 03:52:56.290086 26476 leader_election.cc:302] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c97b68889234966ba5fd2291e647d4d, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.290175 26803 raft_consensus.cc:2781] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:56.290377 26803 raft_consensus.cc:686] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 1 LEADER]: Becoming Leader. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Running, Role: LEADER
I20260812 03:52:56.290485 26803 consensus_queue.cc:237] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } }
I20260812 03:52:56.291211 26214 catalog_manager.cc:5495] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fe5d67f11bd3404aab41649646e08a1e (127.17.54.194). New cstate: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } health_report { overall_health: HEALTHY } } }
I20260812 03:52:56.299346 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.296826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.17.54.194:43173)
0812 03:52:56.296873 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.296875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.296887 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:56.296940 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.296941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.296958 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.297051 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:56.297199 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.297666 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.298538 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.298749 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.298835 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.299019 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.299119 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.299200 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.299249 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.299270 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:52:56.301090 26711 raft_consensus.cc:1261] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Refusing update from remote peer fe5d67f11bd3404aab41649646e08a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.301090 26391 raft_consensus.cc:1261] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Refusing update from remote peer fe5d67f11bd3404aab41649646e08a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.301455 26803 consensus_queue.cc:1035] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.301592 26807 consensus_queue.cc:1035] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.303988 26810 mvcc.cc:205] Tried to move back new op lower bound from 7317531755726307328 to 7317531755686207488. Current Snapshot: MvccSnapshot[applied={T|T < 7317531755726307328}]
I20260812 03:52:56.304433 26811 mvcc.cc:205] Tried to move back new op lower bound from 7317531755726307328 to 7317531755686207488. Current Snapshot: MvccSnapshot[applied={T|T < 7317531755726307328}]
I20260812 03:52:56.306731 26215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:56.306823 26215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:56.307631 26215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: Sending DeleteTablet for 3 replicas of tablet 51deb01f19b84dae8bb4af7e7eb340e9
I20260812 03:52:56.307937 26687 tablet_service.cc:1514] Processing DeleteTablet for tablet 51deb01f19b84dae8bb4af7e7eb340e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43450
I20260812 03:52:56.307974 26370 tablet_service.cc:1514] Processing DeleteTablet for tablet 51deb01f19b84dae8bb4af7e7eb340e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:49968
I20260812 03:52:56.307963 26529 tablet_service.cc:1514] Processing DeleteTablet for tablet 51deb01f19b84dae8bb4af7e7eb340e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:33434
I20260812 03:52:56.308365 26824 tablet_replica.cc:330] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: stopping tablet replica
I20260812 03:52:56.308529 26824 raft_consensus.cc:2229] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:56.308553 26215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:52:56.308640 26824 raft_consensus.cc:2258] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:52:56.308696 26215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:56.308790 26822 tablet_replica.cc:330] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: stopping tablet replica
I20260812 03:52:56.308876 26822 raft_consensus.cc:2229] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:56.308954 26822 raft_consensus.cc:2258] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.309928 26824 ts_tablet_manager.cc:1905] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.310532 26370 tablet_service.cc:1467] Processing CreateTablet for tablet 68e49dc2354141eb86820368518fb1f1 (DEFAULT_TABLE table=test-table [id=75655424e03f4c62a3e87b452b533fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.310658 26370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68e49dc2354141eb86820368518fb1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.311905 26797 tablet_bootstrap.cc:492] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: Bootstrap starting.
I20260812 03:52:56.312055 26687 tablet_service.cc:1467] Processing CreateTablet for tablet 68e49dc2354141eb86820368518fb1f1 (DEFAULT_TABLE table=test-table [id=75655424e03f4c62a3e87b452b533fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.312160 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68e49dc2354141eb86820368518fb1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.312364 26797 tablet_bootstrap.cc:654] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.313069 26797 tablet_bootstrap.cc:492] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: No bootstrap required, opened a new log
I20260812 03:52:56.313118 26797 ts_tablet_manager.cc:1397] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260812 03:52:56.313231 26795 tablet_bootstrap.cc:492] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: Bootstrap starting.
I20260812 03:52:56.313284 26797 raft_consensus.cc:348] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.313346 26797 raft_consensus.cc:374] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.313366 26797 raft_consensus.cc:729] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c97b68889234966ba5fd2291e647d4d, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.313421 26797 consensus_queue.cc:260] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.313503 26797 ts_tablet_manager.cc:1428] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.313496 26529 tablet_service.cc:1467] Processing CreateTablet for tablet 68e49dc2354141eb86820368518fb1f1 (DEFAULT_TABLE table=test-table [id=75655424e03f4c62a3e87b452b533fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.313638 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68e49dc2354141eb86820368518fb1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.313969 26795 tablet_bootstrap.cc:654] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.314778 26795 tablet_bootstrap.cc:492] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: No bootstrap required, opened a new log
I20260812 03:52:56.314834 26795 ts_tablet_manager.cc:1397] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:52:56.314980 26795 raft_consensus.cc:348] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.315068 26795 raft_consensus.cc:374] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.315089 26795 raft_consensus.cc:729] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab02d3d36ff34c3f848c500563938e13, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.315135 26795 consensus_queue.cc:260] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.315243 26795 ts_tablet_manager.cc:1428] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:52:56.315267 26796 tablet_bootstrap.cc:492] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: Bootstrap starting.
I20260812 03:52:56.315766 26796 tablet_bootstrap.cc:654] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.316277 26822 ts_tablet_manager.cc:1905] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.316560 26796 tablet_bootstrap.cc:492] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: No bootstrap required, opened a new log
I20260812 03:52:56.316609 26796 ts_tablet_manager.cc:1397] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:56.316758 26796 raft_consensus.cc:348] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.316818 26796 raft_consensus.cc:374] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.316838 26796 raft_consensus.cc:729] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.316879 26796 consensus_queue.cc:260] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.316964 26796 ts_tablet_manager.cc:1428] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.317760 26824 ts_tablet_manager.cc:1918] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:56.317827 26824 log.cc:1196] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/51deb01f19b84dae8bb4af7e7eb340e9
I20260812 03:52:56.317965 26822 ts_tablet_manager.cc:1918] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:56.318046 26822 log.cc:1196] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/51deb01f19b84dae8bb4af7e7eb340e9
I20260812 03:52:56.318177 26824 ts_tablet_manager.cc:1939] T 51deb01f19b84dae8bb4af7e7eb340e9 P 4c97b68889234966ba5fd2291e647d4d: Deleting consensus metadata
I20260812 03:52:56.318341 26822 ts_tablet_manager.cc:1939] T 51deb01f19b84dae8bb4af7e7eb340e9 P fe5d67f11bd3404aab41649646e08a1e: Deleting consensus metadata
I20260812 03:52:56.318714 26196 catalog_manager.cc:4841] TS 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971): tablet 51deb01f19b84dae8bb4af7e7eb340e9 (table test-table [id=f13bcc0bf41648a184be1dca4f2f256a]) successfully deleted
I20260812 03:52:56.318851 26821 tablet_replica.cc:330] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: stopping tablet replica
I20260812 03:52:56.318853 26194 catalog_manager.cc:4841] TS fe5d67f11bd3404aab41649646e08a1e (127.17.54.194:43173): tablet 51deb01f19b84dae8bb4af7e7eb340e9 (table test-table [id=f13bcc0bf41648a184be1dca4f2f256a]) successfully deleted
I20260812 03:52:56.318979 26821 raft_consensus.cc:2229] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:56.319113 26821 raft_consensus.cc:2258] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.319607 26821 ts_tablet_manager.cc:1905] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.321125 26821 ts_tablet_manager.cc:1918] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:56.321185 26821 log.cc:1196] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/51deb01f19b84dae8bb4af7e7eb340e9
I20260812 03:52:56.321432 26821 ts_tablet_manager.cc:1939] T 51deb01f19b84dae8bb4af7e7eb340e9 P ab02d3d36ff34c3f848c500563938e13: Deleting consensus metadata
I20260812 03:52:56.321883 26196 catalog_manager.cc:4841] TS ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263): tablet 51deb01f19b84dae8bb4af7e7eb340e9 (table test-table [id=f13bcc0bf41648a184be1dca4f2f256a]) successfully deleted
I20260812 03:52:56.328958 26806 raft_consensus.cc:482] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:56.329054 26806 raft_consensus.cc:504] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.329391 26806 leader_election.cc:288] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe5d67f11bd3404aab41649646e08a1e (127.17.54.194:43173), ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.333104 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.329588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43173 (local address 127.17.54.193:45885)
0812 03:52:56.329636 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.329651 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.329702 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.329922 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.329923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.329925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:56.329998 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.330000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.330786 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.330788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.332005 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.332008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.332798 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.332799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.332897 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.332903 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.332967 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.333021 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 03:52:56.333108 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.329713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:45885 (local address 127.17.54.194:43173)
0812 03:52:56.329760 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.329762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.329772 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:52:56.329800 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.329801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.329811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.329867 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:56.330064 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.330688 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.332109 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.332701 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.332780 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.332792 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.332989 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.333039 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:52:56.333184 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.329713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:37065 (local address 127.17.54.195:35263)
0812 03:52:56.329762 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.329764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.329830 (+ 66us) server_negotiation.cc:407] Connection header received
0812 03:52:56.329892 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.329893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.329902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.329959 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:52:56.330153 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.330927 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.332228 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.332762 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.332804 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.332811 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.333073 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.333114 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:56.333392 26553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68e49dc2354141eb86820368518fb1f1" candidate_uuid: "4c97b68889234966ba5fd2291e647d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe5d67f11bd3404aab41649646e08a1e" is_pre_election: true
I20260812 03:52:56.333487 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.329596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:35263 (local address 127.17.54.193:37065)
0812 03:52:56.329782 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.329791 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.329841 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.329979 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.329980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.329984 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:52:56.330069 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.330071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.331022 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.331023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.332137 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.332138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.332895 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.332896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.332975 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.332980 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.333329 (+ 349us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.333385 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":40,"thread_start_us":75,"threads_started":1}
I20260812 03:52:56.333519 26553 raft_consensus.cc:2454] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c97b68889234966ba5fd2291e647d4d in term 0.
I20260812 03:52:56.333788 26314 leader_election.cc:302] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c97b68889234966ba5fd2291e647d4d, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.333912 26806 raft_consensus.cc:2781] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:56.333945 26806 raft_consensus.cc:482] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:56.333825 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68e49dc2354141eb86820368518fb1f1" candidate_uuid: "4c97b68889234966ba5fd2291e647d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13" is_pre_election: true
I20260812 03:52:56.333964 26806 raft_consensus.cc:3037] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.333983 26711 raft_consensus.cc:2454] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c97b68889234966ba5fd2291e647d4d in term 0.
I20260812 03:52:56.334931 26806 raft_consensus.cc:504] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.335078 26806 leader_election.cc:288] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [CANDIDATE]: Term 1 election: Requested vote from peers fe5d67f11bd3404aab41649646e08a1e (127.17.54.194:43173), ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.335358 26553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68e49dc2354141eb86820368518fb1f1" candidate_uuid: "4c97b68889234966ba5fd2291e647d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe5d67f11bd3404aab41649646e08a1e"
I20260812 03:52:56.335448 26553 raft_consensus.cc:3037] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.336248 26553 raft_consensus.cc:2454] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c97b68889234966ba5fd2291e647d4d in term 1.
I20260812 03:52:56.336387 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68e49dc2354141eb86820368518fb1f1" candidate_uuid: "4c97b68889234966ba5fd2291e647d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13"
I20260812 03:52:56.336474 26711 raft_consensus.cc:3037] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.336458 26314 leader_election.cc:302] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c97b68889234966ba5fd2291e647d4d, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.336580 26806 raft_consensus.cc:2781] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:56.336750 26806 raft_consensus.cc:686] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 1 LEADER]: Becoming Leader. State: Replica: 4c97b68889234966ba5fd2291e647d4d, State: Running, Role: LEADER
I20260812 03:52:56.336819 26806 consensus_queue.cc:237] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.337267 26711 raft_consensus.cc:2454] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c97b68889234966ba5fd2291e647d4d in term 1.
I20260812 03:52:56.337529 26215 catalog_manager.cc:5495] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c97b68889234966ba5fd2291e647d4d (127.17.54.193). New cstate: current_term: 1 leader_uuid: "4c97b68889234966ba5fd2291e647d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506775, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.254@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
W20260812 03:52:56.361753 26445 tablet.cc:2369] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:56.363588 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.355180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52866 (local address 127.17.54.254:41863)
0812 03:52:56.355247 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.355250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.355382 (+ 132us) server_negotiation.cc:407] Connection header received
0812 03:52:56.355454 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.355455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.355467 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.355571 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.356082 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.356586 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.357560 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.357899 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.357978 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.360830 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:56.360831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:56.360832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:56.360839 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:56.362287 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.362705 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.362707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.362707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.362766 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.362966 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.362967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.362967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.363084 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:56.363162 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.363317 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.363405 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:52:56.363603 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.355124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:41863 (local address 127.0.0.1:52866)
0812 03:52:56.355291 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.355302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.355398 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.355579 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.355580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.355912 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:56.355964 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.355966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.356674 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.356675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.357458 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.357462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.358036 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.358037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.358103 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.358841 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:56.358847 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:56.360737 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:56.362403 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.362411 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.362414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.362628 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.362834 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:56.362835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:56.362835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:56.362894 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:56.363152 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:56.363160 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:56.363207 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.363293 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.363481 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":66,"thread_start_us":90,"threads_started":1}
I20260812 03:52:56.365767 26215 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:52866:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:52:56.366007 26215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:56.367450 26370 tablet_service.cc:1467] Processing CreateTablet for tablet 34dceb2829d14ed291e542729fdd87b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=754b50868e51468386976f15a86afe1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:56.367477 26687 tablet_service.cc:1467] Processing CreateTablet for tablet 34dceb2829d14ed291e542729fdd87b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=754b50868e51468386976f15a86afe1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:56.367488 26529 tablet_service.cc:1467] Processing CreateTablet for tablet 34dceb2829d14ed291e542729fdd87b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=754b50868e51468386976f15a86afe1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:52:56.367620 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34dceb2829d14ed291e542729fdd87b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.367620 26370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34dceb2829d14ed291e542729fdd87b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.367638 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34dceb2829d14ed291e542729fdd87b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.369107 26796 tablet_bootstrap.cc:492] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: Bootstrap starting.
I20260812 03:52:56.369184 26795 tablet_bootstrap.cc:492] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: Bootstrap starting.
I20260812 03:52:56.369494 26796 tablet_bootstrap.cc:654] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.369645 26795 tablet_bootstrap.cc:654] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.369845 26797 tablet_bootstrap.cc:492] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: Bootstrap starting.
I20260812 03:52:56.370252 26797 tablet_bootstrap.cc:654] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.370538 26795 tablet_bootstrap.cc:492] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: No bootstrap required, opened a new log
I20260812 03:52:56.370595 26795 ts_tablet_manager.cc:1397] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:56.370748 26795 raft_consensus.cc:348] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.370817 26796 tablet_bootstrap.cc:492] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: No bootstrap required, opened a new log
I20260812 03:52:56.370832 26795 raft_consensus.cc:374] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.370864 26795 raft_consensus.cc:729] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab02d3d36ff34c3f848c500563938e13, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.370860 26796 ts_tablet_manager.cc:1397] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:52:56.370903 26795 consensus_queue.cc:260] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.371030 26795 ts_tablet_manager.cc:1428] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:52:56.370994 26796 raft_consensus.cc:348] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.371064 26796 raft_consensus.cc:374] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.371084 26796 raft_consensus.cc:729] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.371133 26796 consensus_queue.cc:260] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.371165 26802 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.371224 26796 ts_tablet_manager.cc:1428] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.371253 26802 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:56.371434 26816 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.371537 26816 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:56.371702 26797 tablet_bootstrap.cc:492] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: No bootstrap required, opened a new log
I20260812 03:52:56.371745 26797 ts_tablet_manager.cc:1397] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:52:56.371891 26797 raft_consensus.cc:348] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.371954 26797 raft_consensus.cc:374] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.371971 26797 raft_consensus.cc:729] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c97b68889234966ba5fd2291e647d4d, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.372015 26797 consensus_queue.cc:260] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.372095 26797 ts_tablet_manager.cc:1428] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.372216 26806 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.372275 26806 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:56.431722 26816 raft_consensus.cc:482] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:56.431847 26816 raft_consensus.cc:504] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.432036 26816 leader_election.cc:288] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971), ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.432298 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34dceb2829d14ed291e542729fdd87b1" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13" is_pre_election: true
I20260812 03:52:56.432277 26391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34dceb2829d14ed291e542729fdd87b1" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c97b68889234966ba5fd2291e647d4d" is_pre_election: true
I20260812 03:52:56.432385 26391 raft_consensus.cc:2454] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 0.
I20260812 03:52:56.432403 26711 raft_consensus.cc:2454] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 0.
I20260812 03:52:56.432565 26476 leader_election.cc:302] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c97b68889234966ba5fd2291e647d4d, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.432657 26816 raft_consensus.cc:2781] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:56.432703 26816 raft_consensus.cc:482] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:56.432734 26816 raft_consensus.cc:3037] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.433434 26816 raft_consensus.cc:504] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.433591 26816 leader_election.cc:288] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 election: Requested vote from peers 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971), ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.433768 26391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34dceb2829d14ed291e542729fdd87b1" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c97b68889234966ba5fd2291e647d4d"
I20260812 03:52:56.433772 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34dceb2829d14ed291e542729fdd87b1" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13"
I20260812 03:52:56.433858 26391 raft_consensus.cc:3037] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.433879 26711 raft_consensus.cc:3037] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.434643 26711 raft_consensus.cc:2454] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 1.
I20260812 03:52:56.434788 26391 raft_consensus.cc:2454] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 1.
I20260812 03:52:56.434821 26476 leader_election.cc:302] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab02d3d36ff34c3f848c500563938e13, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.434916 26816 raft_consensus.cc:2781] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:56.434978 26816 raft_consensus.cc:686] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [term 1 LEADER]: Becoming Leader. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Running, Role: LEADER
I20260812 03:52:56.435021 26816 consensus_queue.cc:237] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.435351 26807 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: TxnStatusTablet state changed. Reason: New leader fe5d67f11bd3404aab41649646e08a1e. Latest consensus state: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.435477 26807 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:56.435684 26842 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:52:56.435676 26215 catalog_manager.cc:5495] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fe5d67f11bd3404aab41649646e08a1e (127.17.54.194). New cstate: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:56.435801 26842 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:52:56.447359 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.444833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43173 (local address 127.0.0.1:33454)
0812 03:52:56.444932 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.444945 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.444995 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.445166 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.445167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.445168 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:56.445221 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.445223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.445878 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.445879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.446534 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.446536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.446921 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.446922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.446990 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.447005 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.447190 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.447197 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.447241 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.447291 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:52:56.447343 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.444897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.17.54.194:43173)
0812 03:52:56.444962 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.444964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.444976 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:52:56.445087 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.445088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.445100 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.445158 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:56.445284 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.445784 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.446619 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.446830 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.446919 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.447093 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.447131 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.447184 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.447244 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.447266 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:52:56.448465 26711 raft_consensus.cc:1261] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Refusing update from remote peer fe5d67f11bd3404aab41649646e08a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.448472 26391 raft_consensus.cc:1261] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Refusing update from remote peer fe5d67f11bd3404aab41649646e08a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.448701 26807 consensus_queue.cc:1035] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.448829 26816 consensus_queue.cc:1035] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.449451 26802 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: TxnStatusTablet state changed. Reason: New leader fe5d67f11bd3404aab41649646e08a1e. Latest consensus state: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.449470 26806 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: TxnStatusTablet state changed. Reason: New leader fe5d67f11bd3404aab41649646e08a1e. Latest consensus state: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.449556 26806 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:56.449553 26802 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:56.450352 26807 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.450433 26807 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:56.450423 26802 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.450502 26802 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P ab02d3d36ff34c3f848c500563938e13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:56.450742 26812 mvcc.cc:205] Tried to move back new op lower bound from 7317531756330823680 to 7317531756278038528. Current Snapshot: MvccSnapshot[applied={T|T < 7317531756330823680 or (T in {7317531756330823680})}]
I20260812 03:52:56.451673 26806 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.451756 26806 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P 4c97b68889234966ba5fd2291e647d4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:52:56.450275 26816 tablet_replica.cc:439] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } } }
I20260812 03:52:56.451913 26816 tablet_replica.cc:442] T 34dceb2829d14ed291e542729fdd87b1 P fe5d67f11bd3404aab41649646e08a1e: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:52:56.455148 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.452431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.17.54.193:38971)
0812 03:52:56.452478 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.452480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.452489 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:56.452570 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.452571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.452590 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.452706 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:56.452877 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.453431 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.454245 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.454539 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.454618 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.454789 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.454906 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.454983 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.455046 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.455067 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:52:56.460330 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.457606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:49663 (local address 127.17.54.194:43173)
0812 03:52:56.457667 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.457669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.457834 (+ 165us) server_negotiation.cc:407] Connection header received
0812 03:52:56.457889 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.457890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.457902 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.457947 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:56.458074 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.458588 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.459690 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.459864 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.459908 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.460116 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.460140 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.460178 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.460263 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.460281 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:52:56.460408 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.457523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43173 (local address 127.17.54.193:49663)
0812 03:52:56.457754 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.457766 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.457832 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.457950 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.457951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.457952 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:56.458018 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.458020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.458667 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.458667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.459602 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.459604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.459938 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.459939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.460011 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.460033 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.460204 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.460207 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.460262 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.460308 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
I20260812 03:52:56.462836 26711 raft_consensus.cc:1261] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Refusing update from remote peer 4c97b68889234966ba5fd2291e647d4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.462958 26553 raft_consensus.cc:1261] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Refusing update from remote peer 4c97b68889234966ba5fd2291e647d4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.463073 26806 consensus_queue.cc:1035] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.463539 26830 consensus_queue.cc:1035] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.472621 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.470128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:38971 (local address 127.17.54.194:49637)
0812 03:52:56.470202 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.470215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.470256 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.470395 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.470396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.470397 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:56.470446 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.470448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.471059 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.471060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.471715 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.471717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.472176 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.472177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.472245 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.472262 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.472481 (+ 219us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.472484 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.472527 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.472567 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:52:56.472635 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.470204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:49637 (local address 127.17.54.193:38971)
0812 03:52:56.470257 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.470259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.470268 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:56.470336 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.470336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.470346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.470395 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:56.470494 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.470973 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.471812 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.472091 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.472161 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.472366 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.472414 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.472466 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.472544 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.472566 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:52:56.482331 26215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:56.482453 26215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:56.483116 26215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: Sending DeleteTablet for 3 replicas of tablet 68e49dc2354141eb86820368518fb1f1
I20260812 03:52:56.483436 26370 tablet_service.cc:1514] Processing DeleteTablet for tablet 68e49dc2354141eb86820368518fb1f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:49968
I20260812 03:52:56.483695 26824 tablet_replica.cc:330] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: stopping tablet replica
I20260812 03:52:56.483769 26824 raft_consensus.cc:2229] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:56.483794 26687 tablet_service.cc:1514] Processing DeleteTablet for tablet 68e49dc2354141eb86820368518fb1f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43450
I20260812 03:52:56.483856 26824 raft_consensus.cc:2258] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.483902 26821 tablet_replica.cc:330] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: stopping tablet replica
I20260812 03:52:56.483964 26821 raft_consensus.cc:2229] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:56.484004 26821 raft_consensus.cc:2258] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.484015 26529 tablet_service.cc:1514] Processing DeleteTablet for tablet 68e49dc2354141eb86820368518fb1f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:33434
I20260812 03:52:56.484100 26822 tablet_replica.cc:330] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: stopping tablet replica
I20260812 03:52:56.484144 26822 raft_consensus.cc:2229] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:56.484174 26822 raft_consensus.cc:2258] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.484380 26821 ts_tablet_manager.cc:1905] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.484696 26824 ts_tablet_manager.cc:1905] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.486092 26821 ts_tablet_manager.cc:1918] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:56.486145 26821 log.cc:1196] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/68e49dc2354141eb86820368518fb1f1
I20260812 03:52:56.486418 26821 ts_tablet_manager.cc:1939] T 68e49dc2354141eb86820368518fb1f1 P ab02d3d36ff34c3f848c500563938e13: Deleting consensus metadata
I20260812 03:52:56.486652 26824 ts_tablet_manager.cc:1918] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:56.486714 26824 log.cc:1196] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/68e49dc2354141eb86820368518fb1f1
I20260812 03:52:56.486938 26824 ts_tablet_manager.cc:1939] T 68e49dc2354141eb86820368518fb1f1 P 4c97b68889234966ba5fd2291e647d4d: Deleting consensus metadata
I20260812 03:52:56.487100 26196 catalog_manager.cc:4841] TS ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263): tablet 68e49dc2354141eb86820368518fb1f1 (table test-table [id=75655424e03f4c62a3e87b452b533fce]) successfully deleted
I20260812 03:52:56.489493 26196 catalog_manager.cc:4841] TS 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971): tablet 68e49dc2354141eb86820368518fb1f1 (table test-table [id=75655424e03f4c62a3e87b452b533fce]) successfully deleted
I20260812 03:52:56.490585 26822 ts_tablet_manager.cc:1905] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.492318 26822 ts_tablet_manager.cc:1918] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:56.492384 26822 log.cc:1196] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/68e49dc2354141eb86820368518fb1f1
I20260812 03:52:56.492619 26822 ts_tablet_manager.cc:1939] T 68e49dc2354141eb86820368518fb1f1 P fe5d67f11bd3404aab41649646e08a1e: Deleting consensus metadata
I20260812 03:52:56.493029 26194 catalog_manager.cc:4841] TS fe5d67f11bd3404aab41649646e08a1e (127.17.54.194:43173): tablet 68e49dc2354141eb86820368518fb1f1 (table test-table [id=75655424e03f4c62a3e87b452b533fce]) successfully deleted
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:52:56.518870 26291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:56.511595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52878 (local address 127.17.54.254:41863)
0812 03:52:56.511660 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.511662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.511822 (+ 160us) server_negotiation.cc:407] Connection header received
0812 03:52:56.511885 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.511886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.511900 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.511961 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.512713 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.513508 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.514715 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.515024 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.515099 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.518797 (+ 3698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52878: BlockingRecv error: recv got EOF from 127.0.0.1:52878 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM
I20260812 03:52:56.529078 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.520012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52882 (local address 127.17.54.254:41863)
0812 03:52:56.520056 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.520058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.520133 (+ 75us) server_negotiation.cc:407] Connection header received
0812 03:52:56.520199 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.520199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.520211 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.520277 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:56.520877 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.521652 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.522773 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.523097 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.523168 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.526008 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:56.526010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:56.526010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:56.526019 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:56.527755 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.528179 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.528180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.528181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.528244 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:56.528482 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:56.528483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:56.528484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:56.528629 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:56.528730 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.528893 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.528976 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:52:56.530845 26215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:56.531035 26215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:56.532605 26370 tablet_service.cc:1467] Processing CreateTablet for tablet eac5b47fbd2e44358ca8c50792b4a86d (DEFAULT_TABLE table=test-table [id=32774f2b34014bf39d773abfb4147074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.532629 26687 tablet_service.cc:1467] Processing CreateTablet for tablet eac5b47fbd2e44358ca8c50792b4a86d (DEFAULT_TABLE table=test-table [id=32774f2b34014bf39d773abfb4147074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.532754 26370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac5b47fbd2e44358ca8c50792b4a86d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.532766 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac5b47fbd2e44358ca8c50792b4a86d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.534150 26529 tablet_service.cc:1467] Processing CreateTablet for tablet eac5b47fbd2e44358ca8c50792b4a86d (DEFAULT_TABLE table=test-table [id=32774f2b34014bf39d773abfb4147074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.534272 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac5b47fbd2e44358ca8c50792b4a86d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.534371 26797 tablet_bootstrap.cc:492] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: Bootstrap starting.
I20260812 03:52:56.534791 26797 tablet_bootstrap.cc:654] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.535135 26795 tablet_bootstrap.cc:492] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: Bootstrap starting.
I20260812 03:52:56.535669 26795 tablet_bootstrap.cc:654] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.537544 26796 tablet_bootstrap.cc:492] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: Bootstrap starting.
I20260812 03:52:56.537894 26796 tablet_bootstrap.cc:654] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.538215 26797 tablet_bootstrap.cc:492] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: No bootstrap required, opened a new log
I20260812 03:52:56.538265 26797 ts_tablet_manager.cc:1397] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 03:52:56.538470 26797 raft_consensus.cc:348] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.538555 26797 raft_consensus.cc:374] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.538578 26797 raft_consensus.cc:729] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c97b68889234966ba5fd2291e647d4d, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.538623 26797 consensus_queue.cc:260] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.538722 26797 ts_tablet_manager.cc:1428] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.538718 26796 tablet_bootstrap.cc:492] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: No bootstrap required, opened a new log
I20260812 03:52:56.538763 26796 ts_tablet_manager.cc:1397] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:56.538862 26796 raft_consensus.cc:348] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.538924 26796 raft_consensus.cc:374] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.538944 26796 raft_consensus.cc:729] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.538988 26796 consensus_queue.cc:260] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.539083 26796 ts_tablet_manager.cc:1428] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.540182 26795 tablet_bootstrap.cc:492] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: No bootstrap required, opened a new log
I20260812 03:52:56.540243 26795 ts_tablet_manager.cc:1397] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260812 03:52:56.540397 26795 raft_consensus.cc:348] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.540475 26795 raft_consensus.cc:374] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.540498 26795 raft_consensus.cc:729] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab02d3d36ff34c3f848c500563938e13, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.540570 26795 consensus_queue.cc:260] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.540679 26795 ts_tablet_manager.cc:1428] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:52:56.647471 26807 raft_consensus.cc:482] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:56.647575 26807 raft_consensus.cc:504] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.647727 26807 leader_election.cc:288] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971), ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.648000 26391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac5b47fbd2e44358ca8c50792b4a86d" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c97b68889234966ba5fd2291e647d4d" is_pre_election: true
I20260812 03:52:56.648013 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac5b47fbd2e44358ca8c50792b4a86d" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13" is_pre_election: true
I20260812 03:52:56.648115 26711 raft_consensus.cc:2454] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 0.
I20260812 03:52:56.648120 26391 raft_consensus.cc:2454] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 0.
I20260812 03:52:56.648365 26476 leader_election.cc:302] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab02d3d36ff34c3f848c500563938e13, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.648474 26807 raft_consensus.cc:2781] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:56.648518 26807 raft_consensus.cc:482] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:56.648540 26807 raft_consensus.cc:3037] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.649260 26807 raft_consensus.cc:504] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.649399 26807 leader_election.cc:288] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 election: Requested vote from peers 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971), ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.649597 26391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac5b47fbd2e44358ca8c50792b4a86d" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c97b68889234966ba5fd2291e647d4d"
I20260812 03:52:56.649639 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac5b47fbd2e44358ca8c50792b4a86d" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13"
I20260812 03:52:56.649701 26391 raft_consensus.cc:3037] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.649703 26711 raft_consensus.cc:3037] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.650388 26711 raft_consensus.cc:2454] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 1.
I20260812 03:52:56.650452 26391 raft_consensus.cc:2454] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 1.
I20260812 03:52:56.650597 26476 leader_election.cc:302] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab02d3d36ff34c3f848c500563938e13, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.650713 26807 raft_consensus.cc:2781] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:56.650770 26807 raft_consensus.cc:686] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 1 LEADER]: Becoming Leader. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Running, Role: LEADER
I20260812 03:52:56.650851 26807 consensus_queue.cc:237] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.651553 26214 catalog_manager.cc:5495] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fe5d67f11bd3404aab41649646e08a1e (127.17.54.194). New cstate: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:56.658849 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.656558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.17.54.194:43173)
0812 03:52:56.656608 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.656610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.656651 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:52:56.656706 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.656707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.656716 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.656762 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:56.656866 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.657344 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.658096 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.658290 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.658461 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.658598 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.658628 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.658669 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.658734 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.658753 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:52:56.659746 26711 raft_consensus.cc:1261] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Refusing update from remote peer fe5d67f11bd3404aab41649646e08a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.659754 26391 raft_consensus.cc:1261] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Refusing update from remote peer fe5d67f11bd3404aab41649646e08a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.659967 26807 consensus_queue.cc:1035] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.660070 26803 consensus_queue.cc:1035] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.662393 26215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:56.662492 26215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:56.663204 26215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: Sending DeleteTablet for 3 replicas of tablet eac5b47fbd2e44358ca8c50792b4a86d
I20260812 03:52:56.663439 26370 tablet_service.cc:1514] Processing DeleteTablet for tablet eac5b47fbd2e44358ca8c50792b4a86d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:49968
I20260812 03:52:56.663475 26687 tablet_service.cc:1514] Processing DeleteTablet for tablet eac5b47fbd2e44358ca8c50792b4a86d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43450
I20260812 03:52:56.663511 26529 tablet_service.cc:1514] Processing DeleteTablet for tablet eac5b47fbd2e44358ca8c50792b4a86d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:33434
I20260812 03:52:56.663509 26369 tablet_service.cc:1514] Processing DeleteTablet for tablet eac5b47fbd2e44358ca8c50792b4a86d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:49968
I20260812 03:52:56.663551 26824 tablet_replica.cc:330] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: stopping tablet replica
I20260812 03:52:56.663568 26821 tablet_replica.cc:330] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: stopping tablet replica
I20260812 03:52:56.663599 26824 raft_consensus.cc:2229] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:56.663614 26821 raft_consensus.cc:2229] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:56.663643 26824 raft_consensus.cc:2258] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.663656 26821 raft_consensus.cc:2258] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.663738 26822 tablet_replica.cc:330] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: stopping tablet replica
I20260812 03:52:56.663794 26822 raft_consensus.cc:2229] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:56.663851 26822 raft_consensus.cc:2258] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.663998 26824 ts_tablet_manager.cc:1905] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.664114 26822 ts_tablet_manager.cc:1905] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.664137 26821 ts_tablet_manager.cc:1905] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.665613 26821 ts_tablet_manager.cc:1918] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:56.665647 26822 ts_tablet_manager.cc:1918] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:56.665678 26821 log.cc:1196] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/eac5b47fbd2e44358ca8c50792b4a86d
I20260812 03:52:56.665693 26822 log.cc:1196] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/eac5b47fbd2e44358ca8c50792b4a86d
I20260812 03:52:56.665685 26824 ts_tablet_manager.cc:1918] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:52:56.665738 26824 log.cc:1196] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/eac5b47fbd2e44358ca8c50792b4a86d
I20260812 03:52:56.665663 26215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:52:56.665829 26215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:52:56.665915 26822 ts_tablet_manager.cc:1939] T eac5b47fbd2e44358ca8c50792b4a86d P fe5d67f11bd3404aab41649646e08a1e: Deleting consensus metadata
I20260812 03:52:56.665915 26821 ts_tablet_manager.cc:1939] T eac5b47fbd2e44358ca8c50792b4a86d P ab02d3d36ff34c3f848c500563938e13: Deleting consensus metadata
I20260812 03:52:56.665971 26824 ts_tablet_manager.cc:1939] T eac5b47fbd2e44358ca8c50792b4a86d P 4c97b68889234966ba5fd2291e647d4d: Deleting consensus metadata
I20260812 03:52:56.666360 26194 catalog_manager.cc:4841] TS fe5d67f11bd3404aab41649646e08a1e (127.17.54.194:43173): tablet eac5b47fbd2e44358ca8c50792b4a86d (table test-table [id=32774f2b34014bf39d773abfb4147074]) successfully deleted
I20260812 03:52:56.666504 26196 catalog_manager.cc:4841] TS ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263): tablet eac5b47fbd2e44358ca8c50792b4a86d (table test-table [id=32774f2b34014bf39d773abfb4147074]) successfully deleted
I20260812 03:52:56.666559 26196 catalog_manager.cc:4841] TS 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971): tablet eac5b47fbd2e44358ca8c50792b4a86d (table test-table [id=32774f2b34014bf39d773abfb4147074]) successfully deleted
W20260812 03:52:56.666656 26196 catalog_manager.cc:4808] TS 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971): delete failed for tablet eac5b47fbd2e44358ca8c50792b4a86d because the tablet was not found. No further retry: Not found: Tablet not found: eac5b47fbd2e44358ca8c50792b4a86d
I20260812 03:52:56.667085 26369 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc0be51b01c422dbb28fae2883d074d (DEFAULT_TABLE table=test-table [id=54889bed7a904493a97fc61006ed4725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.667098 26529 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc0be51b01c422dbb28fae2883d074d (DEFAULT_TABLE table=test-table [id=54889bed7a904493a97fc61006ed4725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.667236 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc0be51b01c422dbb28fae2883d074d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.667230 26369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc0be51b01c422dbb28fae2883d074d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.668109 26796 tablet_bootstrap.cc:492] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: Bootstrap starting.
I20260812 03:52:56.668148 26687 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc0be51b01c422dbb28fae2883d074d (DEFAULT_TABLE table=test-table [id=54889bed7a904493a97fc61006ed4725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:52:56.668253 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc0be51b01c422dbb28fae2883d074d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:56.668260 26797 tablet_bootstrap.cc:492] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d: Bootstrap starting.
I20260812 03:52:56.668435 26796 tablet_bootstrap.cc:654] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.668571 26797 tablet_bootstrap.cc:654] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.669050 26796 tablet_bootstrap.cc:492] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: No bootstrap required, opened a new log
I20260812 03:52:56.669122 26796 ts_tablet_manager.cc:1397] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:56.669239 26796 raft_consensus.cc:348] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.669312 26796 raft_consensus.cc:374] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.669334 26796 raft_consensus.cc:729] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.669387 26796 consensus_queue.cc:260] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.669440 26795 tablet_bootstrap.cc:492] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: Bootstrap starting.
I20260812 03:52:56.669495 26796 ts_tablet_manager.cc:1428] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.669828 26795 tablet_bootstrap.cc:654] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:56.669835 26797 tablet_bootstrap.cc:492] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d: No bootstrap required, opened a new log
I20260812 03:52:56.669895 26797 ts_tablet_manager.cc:1397] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:52:56.670018 26797 raft_consensus.cc:348] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.670090 26797 raft_consensus.cc:374] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.670118 26797 raft_consensus.cc:729] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c97b68889234966ba5fd2291e647d4d, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.670149 26797 consensus_queue.cc:260] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.670245 26797 ts_tablet_manager.cc:1428] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:56.670567 26795 tablet_bootstrap.cc:492] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: No bootstrap required, opened a new log
I20260812 03:52:56.670634 26795 ts_tablet_manager.cc:1397] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:56.670805 26795 raft_consensus.cc:348] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.670879 26795 raft_consensus.cc:374] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:56.670912 26795 raft_consensus.cc:729] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab02d3d36ff34c3f848c500563938e13, State: Initialized, Role: FOLLOWER
I20260812 03:52:56.670964 26795 consensus_queue.cc:260] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.671065 26795 ts_tablet_manager.cc:1428] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260812 03:52:56.721513 26765 tablet.cc:2369] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:52:56.837387 26807 raft_consensus.cc:482] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:52:56.837495 26807 raft_consensus.cc:504] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.837692 26807 leader_election.cc:288] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971), ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.838006 26391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc0be51b01c422dbb28fae2883d074d" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c97b68889234966ba5fd2291e647d4d" is_pre_election: true
I20260812 03:52:56.838050 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc0be51b01c422dbb28fae2883d074d" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13" is_pre_election: true
I20260812 03:52:56.838150 26391 raft_consensus.cc:2454] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 0.
I20260812 03:52:56.838153 26711 raft_consensus.cc:2454] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 0.
I20260812 03:52:56.838363 26476 leader_election.cc:302] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab02d3d36ff34c3f848c500563938e13, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.838529 26807 raft_consensus.cc:2781] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:52:56.838586 26807 raft_consensus.cc:482] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:52:56.838603 26807 raft_consensus.cc:3037] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.839335 26807 raft_consensus.cc:504] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.839448 26807 leader_election.cc:288] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 election: Requested vote from peers 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971), ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263)
I20260812 03:52:56.839709 26711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc0be51b01c422dbb28fae2883d074d" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab02d3d36ff34c3f848c500563938e13"
I20260812 03:52:56.839705 26391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc0be51b01c422dbb28fae2883d074d" candidate_uuid: "fe5d67f11bd3404aab41649646e08a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c97b68889234966ba5fd2291e647d4d"
I20260812 03:52:56.839792 26711 raft_consensus.cc:3037] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.839792 26391 raft_consensus.cc:3037] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:56.840590 26391 raft_consensus.cc:2454] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 1.
I20260812 03:52:56.840593 26711 raft_consensus.cc:2454] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5d67f11bd3404aab41649646e08a1e in term 1.
I20260812 03:52:56.840822 26476 leader_election.cc:302] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab02d3d36ff34c3f848c500563938e13, fe5d67f11bd3404aab41649646e08a1e; no voters:
I20260812 03:52:56.840934 26807 raft_consensus.cc:2781] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:56.840989 26807 raft_consensus.cc:686] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 1 LEADER]: Becoming Leader. State: Replica: fe5d67f11bd3404aab41649646e08a1e, State: Running, Role: LEADER
I20260812 03:52:56.841078 26807 consensus_queue.cc:237] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } }
I20260812 03:52:56.841769 26215 catalog_manager.cc:5495] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fe5d67f11bd3404aab41649646e08a1e (127.17.54.194). New cstate: current_term: 1 leader_uuid: "fe5d67f11bd3404aab41649646e08a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe5d67f11bd3404aab41649646e08a1e" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 } health_report { overall_health: UNKNOWN } } }
I20260812 03:52:56.856832 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.854650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:44265 (local address 127.17.54.194:43173)
0812 03:52:56.854694 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:56.854696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:56.854739 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:52:56.854811 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:56.854812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:56.854822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:56.854870 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:56.855000 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.855481 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.856189 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.856398 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.856479 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.856597 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.856629 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.856675 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:56.856730 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:52:56.856746 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":17}
I20260812 03:52:56.856832 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.854590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43173 (local address 127.17.54.194:44265)
0812 03:52:56.854693 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.854704 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:56.854755 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:56.854871 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:56.854872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:56.854873 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:52:56.854937 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.854939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.855563 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.855564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.856104 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:56.856105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:56.856475 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:56.856476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:56.856526 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:56.856530 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:52:56.856675 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:52:56.856676 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:52:56.856726 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:52:56.856768 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":6}
I20260812 03:52:56.858795 26391 raft_consensus.cc:1261] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Refusing update from remote peer fe5d67f11bd3404aab41649646e08a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.858807 26711 raft_consensus.cc:1261] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Refusing update from remote peer fe5d67f11bd3404aab41649646e08a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:56.859026 26887 consensus_queue.cc:1035] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c97b68889234966ba5fd2291e647d4d" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.859145 26816 consensus_queue.cc:1035] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab02d3d36ff34c3f848c500563938e13" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 35263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:56.871591 26215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:56.871695 26215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:52:56.872359 26215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19c0cfad41524fe18335d7eabfd47260: Sending DeleteTablet for 3 replicas of tablet 8fc0be51b01c422dbb28fae2883d074d
I20260812 03:52:56.872615 26369 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc0be51b01c422dbb28fae2883d074d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:49968
I20260812 03:52:56.872634 26528 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc0be51b01c422dbb28fae2883d074d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:33434
I20260812 03:52:56.872718 26822 tablet_replica.cc:330] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: stopping tablet replica
I20260812 03:52:56.872766 26822 raft_consensus.cc:2229] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:52:56.872752 26824 tablet_replica.cc:330] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d: stopping tablet replica
I20260812 03:52:56.872824 26824 raft_consensus.cc:2229] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:56.872828 26822 raft_consensus.cc:2258] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.872866 26824 raft_consensus.cc:2258] T 8fc0be51b01c422dbb28fae2883d074d P 4c97b68889234966ba5fd2291e647d4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.872937 26687 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc0be51b01c422dbb28fae2883d074d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:52:56 UTC) from {username='oryx', principal='oryx/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43450
I20260812 03:52:56.873042 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 26295
I20260812 03:52:56.873131 26821 tablet_replica.cc:330] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: stopping tablet replica
I20260812 03:52:56.873188 26821 raft_consensus.cc:2229] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:52:56.873219 26821 raft_consensus.cc:2258] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:52:56.876085 26822 ts_tablet_manager.cc:1905] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.877139 26821 ts_tablet_manager.cc:1905] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:52:56.877638 26822 ts_tablet_manager.cc:1918] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:56.877713 26822 log.cc:1196] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/8fc0be51b01c422dbb28fae2883d074d
I20260812 03:52:56.877952 26822 ts_tablet_manager.cc:1939] T 8fc0be51b01c422dbb28fae2883d074d P fe5d67f11bd3404aab41649646e08a1e: Deleting consensus metadata
I20260812 03:52:56.878336 26194 catalog_manager.cc:4841] TS fe5d67f11bd3404aab41649646e08a1e (127.17.54.194:43173): tablet 8fc0be51b01c422dbb28fae2883d074d (table test-table [id=54889bed7a904493a97fc61006ed4725]) successfully deleted
I20260812 03:52:56.878492 26821 ts_tablet_manager.cc:1918] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:52:56.878568 26821 log.cc:1196] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/8fc0be51b01c422dbb28fae2883d074d
I20260812 03:52:56.878772 26821 ts_tablet_manager.cc:1939] T 8fc0be51b01c422dbb28fae2883d074d P ab02d3d36ff34c3f848c500563938e13: Deleting consensus metadata
I20260812 03:52:56.879171 26196 catalog_manager.cc:4841] TS ab02d3d36ff34c3f848c500563938e13 (127.17.54.195:35263): tablet 8fc0be51b01c422dbb28fae2883d074d (table test-table [id=54889bed7a904493a97fc61006ed4725]) successfully deleted
W20260812 03:52:56.884477 26196 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:38971 (error 108)
I20260812 03:52:56.884845 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 26455
I20260812 03:52:56.885095 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:56.884885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:38971 (local address 127.0.0.1:49992)
0812 03:52:56.885000 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:56.885045 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":45,"threads_started":1}
W20260812 03:52:56.885181 26196 catalog_manager.cc:4584] TS 4c97b68889234966ba5fd2291e647d4d (127.17.54.193:38971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fc0be51b01c422dbb28fae2883d074d: Network error: Client connection negotiation failed: client connection to 127.17.54.193:38971: connect: Connection refused (error 111)
I20260812 03:52:56.894275 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 26617
I20260812 03:52:56.904536 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 26176
2026-08-12T03:52:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2633 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26895](info): set up 2 sockets
Aug 12 03:52:56 dist-test-slave-tm1g krb5kdc[26895](info): commencing operation
krb5kdc: starting...
W20260812 03:52:58.026856 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:58 dist-test-slave-tm1g krb5kdc[26895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:58.162791 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:40409
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:37649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:40409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:58.247628 26909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:58.247905 26909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:58.247946 26909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:58.249195 26909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:58.249269 26909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:58.249291 26909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:58.249312 26909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:58.249338 26909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:58.251021 26909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:37649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:40409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:40409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:58.251341 26909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:58.251595 26909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:58.256960 26915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:58.256970 26918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:58.257001 26916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:58.257298 26909 server_base.cc:1034] running on GCE node
I20260812 03:52:58.257608 26909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:58.257946 26909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:58.259131 26909 hybrid_clock.cc:648] HybridClock initialized: now 1786506778259056 us; error 102 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T03:52:58Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1364 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:58 dist-test-slave-tm1g krb5kdc[26923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:58 dist-test-slave-tm1g krb5kdc[26923](info): set up 2 sockets
Aug 12 03:52:58 dist-test-slave-tm1g krb5kdc[26923](info): commencing operation
krb5kdc: starting...
W20260812 03:52:59.425439 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:52:59 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:52:59.561344 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:36705
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:36705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:59.645704 26938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:59.645928 26938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:59.645953 26938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:59.647202 26938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:59.647253 26938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:59.647267 26938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:59.647280 26938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:59.647292 26938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:59.648823 26938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:36705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:59.649099 26938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:59.649353 26938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:59.654392 26947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:59.654426 26944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:59.654456 26945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:59.654737 26938 server_base.cc:1034] running on GCE node
I20260812 03:52:59.655125 26938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:59.655457 26938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:59.656636 26938 hybrid_clock.cc:648] HybridClock initialized: now 1786506779656631 us; error 38 us; skew 500 ppm
Aug 12 03:52:59 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:59.659376 26938 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:52:59.659792 26938 webserver.cc:460] Webserver started at http://127.17.54.254:46135/ using document root <none> and password file <none>
I20260812 03:52:59.659991 26938 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:59.660032 26938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:59.660122 26938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:59.661181 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "2c0bdf61e65f425380feef307520a59a"
format_stamp: "Formatted at 2026-08-12 03:52:59 on dist-test-slave-tm1g"
server_key: "156367521ff81477f0c913b9da1b1020"
server_key_iv: "7089865d553ca27186b91f5ca9c882bd"
server_key_version: "encryptionkey@0"
I20260812 03:52:59.661506 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "2c0bdf61e65f425380feef307520a59a"
format_stamp: "Formatted at 2026-08-12 03:52:59 on dist-test-slave-tm1g"
server_key: "156367521ff81477f0c913b9da1b1020"
server_key_iv: "7089865d553ca27186b91f5ca9c882bd"
server_key_version: "encryptionkey@0"
I20260812 03:52:59.662966 26938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:59.663780 26956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:59.664070 26938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:59.664194 26938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "2c0bdf61e65f425380feef307520a59a"
format_stamp: "Formatted at 2026-08-12 03:52:59 on dist-test-slave-tm1g"
server_key: "156367521ff81477f0c913b9da1b1020"
server_key_iv: "7089865d553ca27186b91f5ca9c882bd"
server_key_version: "encryptionkey@0"
I20260812 03:52:59.664284 26938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:59.675643 26938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:59.677861 26938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:59.678038 26938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:59.682696 26938 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:36705
I20260812 03:52:59.682777 27010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:36705 every 8 connection(s)
I20260812 03:52:59.683144 26938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:59.684149 27011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:59.686596 27011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Bootstrap starting.
I20260812 03:52:59.687232 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 26938
I20260812 03:52:59.687285 27011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:59.687353 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:52:59.687593 17627 external_mini_cluster.cc:1428] Setting key 3f494d7835d23e5ddae33993f0313a0a
I20260812 03:52:59.687635 27011 log.cc:824] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:59.688599 27011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: No bootstrap required, opened a new log
I20260812 03:52:59.691329 27011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } }
I20260812 03:52:59.691537 27011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:59.691577 27011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0bdf61e65f425380feef307520a59a, State: Initialized, Role: FOLLOWER
I20260812 03:52:59.691715 27011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } }
I20260812 03:52:59.691797 27011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:59.691826 27011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:59.691874 27011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:59.692654 27011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } }
I20260812 03:52:59.692783 27011 leader_election.cc:302] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c0bdf61e65f425380feef307520a59a; no voters:
I20260812 03:52:59.692984 27011 leader_election.cc:288] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:59.693038 27017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:59.693221 27017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 1 LEADER]: Becoming Leader. State: Replica: 2c0bdf61e65f425380feef307520a59a, State: Running, Role: LEADER
I20260812 03:52:59.693341 27011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:59.693378 27017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } }
I20260812 03:52:59.693794 27017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c0bdf61e65f425380feef307520a59a. Latest consensus state: current_term: 1 leader_uuid: "2c0bdf61e65f425380feef307520a59a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } } }
I20260812 03:52:59.694168 27018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c0bdf61e65f425380feef307520a59a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } } }
I20260812 03:52:59.694271 27017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:59.694291 27018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:59.694814 27026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 03:52:59 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:59.695309 27026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:59.697103 27026 catalog_manager.cc:1347] Generated new cluster ID: f247bc9ae45d41478a654606072f0834
I20260812 03:52:59.697149 27026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:59.699868 27016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:59.688177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.17.54.254:36705)
0812 03:52:59.688348 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:59.688356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:59.688385 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:52:59.688920 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:59.688927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:59.688991 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:59.689107 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:59.689999 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:59.690944 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:59.692119 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:59.692526 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:59.692610 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:59.695878 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:59.695883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:59.695884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:59.695896 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:59.697940 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:59.698459 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:59.698463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:59.698463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:59.698579 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:59.698816 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:59.698817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:59.698817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:59.699166 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:59.699356 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:59.699429 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:52:59.699578 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260812 03:52:59.703419 27026 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:59.703934 27026 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:59.707732 27026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Generated new TSK 0
I20260812 03:52:59.707921 27026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:52:59.745304 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36705
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:59.830960 27039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:59.831209 27039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:59.831251 27039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:59.832541 27039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:59.832623 27039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:59.832669 27039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:52:59.834270 27039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:59.834620 27039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:59.834888 27039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:59.840274 27048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:59.840296 27045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:59.840485 27039 server_base.cc:1034] running on GCE node
W20260812 03:52:59.840555 27046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:59.840793 27039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:59.841122 27039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:59.842352 27039 hybrid_clock.cc:648] HybridClock initialized: now 1786506779842298 us; error 36 us; skew 500 ppm
Aug 12 03:52:59 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:59.845140 27039 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:52:59.845702 27039 webserver.cc:460] Webserver started at http://127.17.54.193:32957/ using document root <none> and password file <none>
I20260812 03:52:59.845979 27039 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:59.846052 27039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:59.846213 27039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:59.847357 27039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "4ad8f7e5365b449cac42f3cfede010b3"
format_stamp: "Formatted at 2026-08-12 03:52:59 on dist-test-slave-tm1g"
server_key: "4499016303980b066df9fee9289e8e3f"
server_key_iv: "9ff4fe72011187f0e46e1c5977830a2b"
server_key_version: "encryptionkey@0"
I20260812 03:52:59.847746 27039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "4ad8f7e5365b449cac42f3cfede010b3"
format_stamp: "Formatted at 2026-08-12 03:52:59 on dist-test-slave-tm1g"
server_key: "4499016303980b066df9fee9289e8e3f"
server_key_iv: "9ff4fe72011187f0e46e1c5977830a2b"
server_key_version: "encryptionkey@0"
I20260812 03:52:59.849218 27039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:59.849992 27057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:59.850193 27039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:59.850296 27039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "4ad8f7e5365b449cac42f3cfede010b3"
format_stamp: "Formatted at 2026-08-12 03:52:59 on dist-test-slave-tm1g"
server_key: "4499016303980b066df9fee9289e8e3f"
server_key_iv: "9ff4fe72011187f0e46e1c5977830a2b"
server_key_version: "encryptionkey@0"
I20260812 03:52:59.850418 27039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:59.867853 27039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:59.870381 27039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:59.870543 27039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:59.870795 27039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:52:59.871153 27039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:52:59.871188 27039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:59.871233 27039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:52:59.871259 27039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:59.877259 27039 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:43751
I20260812 03:52:59.877342 27186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:43751 every 8 connection(s)
I20260812 03:52:59.877702 27039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:52:59.881709 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 27039
I20260812 03:52:59.881805 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:52:59.882026 17627 external_mini_cluster.cc:1428] Setting key 6eb32b4929b2212c47d3d4c302b4a415
Aug 12 03:52:59 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:52:59.888808 27016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:59.878532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:48845 (local address 127.17.54.254:36705)
0812 03:52:59.878580 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:59.878582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:59.878843 (+ 261us) server_negotiation.cc:407] Connection header received
0812 03:52:59.879324 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:59.879325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:59.879335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:59.879432 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:59.880449 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:59.880980 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:59.882085 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:59.882422 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:59.882510 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:59.885306 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:59.885307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:59.885308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:59.885315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:59.886947 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:59.887499 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:59.887500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:59.887501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:59.887556 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:59.888017 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:59.888018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:59.888018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:59.888148 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:59.888215 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:59.888645 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 03:52:59.888737 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:52:59.889127 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:59.878494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:48845)
0812 03:52:59.878698 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:59.878744 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:59.879238 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:59.879515 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:59.879519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:59.880142 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:59.880358 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:59.880364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:59.881080 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:59.881081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:59.881979 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:59.881982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:59.882542 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:59.882543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:59.882714 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:59.883269 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:59.883280 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:59.885179 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:59.887038 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:59.887047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:59.887050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:59.887416 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:59.887630 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:59.887631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:59.887631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:59.887786 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:59.888233 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:59.888243 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:59.888450 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 03:52:59.888608 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 03:52:59.888823 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
I20260812 03:52:59.889801 27188 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36705
I20260812 03:52:59.889923 27188 heartbeater.cc:464] Registering TS with master...
I20260812 03:52:59.890143 27188 heartbeater.cc:511] Master 127.17.54.254:36705 requested a full tablet report, sending...
I20260812 03:52:59.890784 26973 ts_manager.cc:194] Registered new tserver with Master: 4ad8f7e5365b449cac42f3cfede010b3 (127.17.54.193:43751)
I20260812 03:52:59.891439 26973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:48845
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:52:59.913803 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36705
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:59.998973 27193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:59.999193 27193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:59.999219 27193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:00.000434 27193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:00.000487 27193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:00.000515 27193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:53:00.002048 27193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:00.002341 27193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:00.002616 27193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:53:00.008391 27193 server_base.cc:1034] running on GCE node
W20260812 03:53:00.008436 27200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.008358 27199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.008369 27203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:00.008925 27193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:00.009228 27193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:00.010391 27193 hybrid_clock.cc:648] HybridClock initialized: now 1786506780010350 us; error 62 us; skew 500 ppm
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:00.013263 27193 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:53:00.013892 27193 webserver.cc:460] Webserver started at http://127.17.54.194:44283/ using document root <none> and password file <none>
I20260812 03:53:00.014206 27193 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:00.014284 27193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:00.014626 27193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:00.015897 27193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "f2eb5ca7f58c4e5d8b6af93fc48a4b83"
format_stamp: "Formatted at 2026-08-12 03:53:00 on dist-test-slave-tm1g"
server_key: "0d75d68b66455c93ec87580aeb179491"
server_key_iv: "e6bbc816c79398e4eed51eec5479164e"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.016328 27193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "f2eb5ca7f58c4e5d8b6af93fc48a4b83"
format_stamp: "Formatted at 2026-08-12 03:53:00 on dist-test-slave-tm1g"
server_key: "0d75d68b66455c93ec87580aeb179491"
server_key_iv: "e6bbc816c79398e4eed51eec5479164e"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.017889 27193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:53:00.018896 27212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.019238 27193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:53:00.019394 27193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "f2eb5ca7f58c4e5d8b6af93fc48a4b83"
format_stamp: "Formatted at 2026-08-12 03:53:00 on dist-test-slave-tm1g"
server_key: "0d75d68b66455c93ec87580aeb179491"
server_key_iv: "e6bbc816c79398e4eed51eec5479164e"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.019484 27193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:00.044215 27193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:00.046838 27193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:00.047036 27193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:00.047317 27193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:00.047658 27193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:00.047711 27193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.047753 27193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:00.047788 27193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.054347 27193 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:41315
I20260812 03:53:00.054458 27341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:41315 every 8 connection(s)
I20260812 03:53:00.054756 27193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:53:00.059975 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 27193
I20260812 03:53:00.060115 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:53:00.060344 17627 external_mini_cluster.cc:1428] Setting key 275ffca14c6f76b9c6ad7220c13dbebb
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:53:00.065732 27016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.055509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:46687 (local address 127.17.54.254:36705)
0812 03:53:00.055556 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.055558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.055855 (+ 297us) server_negotiation.cc:407] Connection header received
0812 03:53:00.056430 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.056431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.056446 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.056490 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.057534 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.058060 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.059194 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.059425 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.059661 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.062168 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:00.062170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:00.062170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:53:00.062179 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:00.064045 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:00.064562 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:00.064563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:00.064564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:00.064673 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:00.065004 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:00.065005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:00.065005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:00.065143 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:00.065223 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:00.065536 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 03:53:00.065634 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:53:00.065944 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.055474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:46687)
0812 03:53:00.055675 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.055744 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.056304 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.056574 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.056578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.057199 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.057423 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.057429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.058163 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.058164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.059097 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.059100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.059669 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.059670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.059829 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.060487 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.060495 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.062063 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:00.064162 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:00.064170 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:00.064172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:00.064450 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:00.064771 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:00.064772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:00.064772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:00.064902 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:00.065211 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:00.065219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:00.065385 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 03:53:00.065498 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:53:00.065684 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260812 03:53:00.066804 27342 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36705
I20260812 03:53:00.066923 27342 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:00.067199 27342 heartbeater.cc:511] Master 127.17.54.254:36705 requested a full tablet report, sending...
I20260812 03:53:00.067669 26973 ts_manager.cc:194] Registered new tserver with Master: f2eb5ca7f58c4e5d8b6af93fc48a4b83 (127.17.54.194:41315)
I20260812 03:53:00.068100 26973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:46687
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:53:00.094049 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36705
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:53:00.177448 27347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:00.177678 27347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:00.177717 27347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:00.178920 27347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:00.178987 27347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:00.179036 27347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:53:00.180558 27347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:00.180757 27347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:00.180940 27347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:00.186023 27357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.186064 27353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.186137 27352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:00.186414 27347 server_base.cc:1034] running on GCE node
I20260812 03:53:00.186681 27347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:00.186956 27347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:00.188153 27347 hybrid_clock.cc:648] HybridClock initialized: now 1786506780188130 us; error 38 us; skew 500 ppm
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:00.190826 27347 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:53:00.191256 27347 webserver.cc:460] Webserver started at http://127.17.54.195:43995/ using document root <none> and password file <none>
I20260812 03:53:00.191520 27347 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:00.191567 27347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:00.191735 27347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:00.192818 27347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "6aa210951e4f4054acbc805621435936"
format_stamp: "Formatted at 2026-08-12 03:53:00 on dist-test-slave-tm1g"
server_key: "324bf8fa55dcc6f1cc6c0e8696aa3c50"
server_key_iv: "f57efcbd829683354d0f92dcce02c58d"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.193212 27347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "6aa210951e4f4054acbc805621435936"
format_stamp: "Formatted at 2026-08-12 03:53:00 on dist-test-slave-tm1g"
server_key: "324bf8fa55dcc6f1cc6c0e8696aa3c50"
server_key_iv: "f57efcbd829683354d0f92dcce02c58d"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.194826 27347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:53:00.195736 27365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.195977 27347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:00.196084 27347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "6aa210951e4f4054acbc805621435936"
format_stamp: "Formatted at 2026-08-12 03:53:00 on dist-test-slave-tm1g"
server_key: "324bf8fa55dcc6f1cc6c0e8696aa3c50"
server_key_iv: "f57efcbd829683354d0f92dcce02c58d"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.196165 27347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:00.218981 27347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:00.220964 27347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:00.221124 27347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:00.221397 27347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:00.221796 27347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:00.221863 27347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.221909 27347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:00.221943 27347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.228251 27347 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:33445
I20260812 03:53:00.228309 27493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:33445 every 8 connection(s)
I20260812 03:53:00.228648 27347 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:53:00.230248 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 27347
I20260812 03:53:00.230388 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:53:00.230592 17627 external_mini_cluster.cc:1428] Setting key 1861d2d07ff6ecdbe64624acbc80167a
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:53:00.238065 27016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.229454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:56363 (local address 127.17.54.254:36705)
0812 03:53:00.229503 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.229505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.229697 (+ 192us) server_negotiation.cc:407] Connection header received
0812 03:53:00.230261 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.230262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.230271 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.230409 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.231593 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.232127 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.233099 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.233285 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.233335 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.235492 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:00.235494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:00.235494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:53:00.235501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:00.236715 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:00.237161 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:00.237162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:00.237163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:00.237221 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:00.237499 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:00.237500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:00.237500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:00.237611 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:00.237665 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:00.237930 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 03:53:00.237997 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:53:00.238334 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.229398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:56363)
0812 03:53:00.229568 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.229611 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.230155 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.230487 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.230492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.231221 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.231428 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.231433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.232209 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.232210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.233022 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.233024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.233359 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.233360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.233460 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.233947 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.233956 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.235405 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:00.236810 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:00.236817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:00.236819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:00.237085 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:00.237308 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:00.237309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:00.237310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:00.237434 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:00.237676 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:00.237681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:00.237804 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 03:53:00.237900 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:53:00.238028 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260812 03:53:00.238801 27494 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36705
I20260812 03:53:00.238902 27494 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:00.239060 27494 heartbeater.cc:511] Master 127.17.54.254:36705 requested a full tablet report, sending...
I20260812 03:53:00.239470 26973 ts_manager.cc:194] Registered new tserver with Master: 6aa210951e4f4054acbc805621435936 (127.17.54.195:33445)
I20260812 03:53:00.239818 26973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:56363
I20260812 03:53:00.242517 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab.
Entry for principal oryx/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab.
WARNING: no policy specified for oryx/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab.
Entry for principal oryx/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab.
WARNING: no policy specified for oryx/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab.
Entry for principal oryx/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab.
I20260812 03:53:00.330188 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 27039
I20260812 03:53:00.338133 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 27193
I20260812 03:53:00.346472 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 27347
I20260812 03:53:00.354815 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 26938
I20260812 03:53:00.361697 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:36705
--webserver_interface=127.17.54.254
--webserver_port=46135
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:36705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:53:00.447404 27505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:00.447623 27505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:00.447646 27505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:00.448873 27505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:53:00.448925 27505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:00.448940 27505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:00.448952 27505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:53:00.448966 27505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:53:00.450580 27505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:36705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=46135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:00.450866 27505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:00.451123 27505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:00.456377 27510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.456393 27513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.456426 27511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:00.456450 27505 server_base.cc:1034] running on GCE node
I20260812 03:53:00.456887 27505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:00.457192 27505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:00.458379 27505 hybrid_clock.cc:648] HybridClock initialized: now 1786506780458358 us; error 37 us; skew 500 ppm
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:00.460992 27505 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:53:00.461386 27505 webserver.cc:460] Webserver started at http://127.17.54.254:46135/ using document root <none> and password file <none>
I20260812 03:53:00.461580 27505 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:00.461630 27505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:00.463043 27505 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:53:00.463697 27522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.463891 27505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:53:00.463981 27505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "2c0bdf61e65f425380feef307520a59a"
format_stamp: "Formatted at 2026-08-12 03:52:59 on dist-test-slave-tm1g"
server_key: "156367521ff81477f0c913b9da1b1020"
server_key_iv: "7089865d553ca27186b91f5ca9c882bd"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.464318 27505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:00.477201 27505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:00.479476 27505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:00.479636 27505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:00.483955 27505 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:36705
I20260812 03:53:00.483989 27583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:36705 every 8 connection(s)
I20260812 03:53:00.484367 27505 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:53:00.485903 27584 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:53:00.486568 27584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Bootstrap starting.
I20260812 03:53:00.487584 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 27505
I20260812 03:53:00.487947 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:43751
--local_ip_for_outbound_sockets=127.17.54.193
--tserver_master_addrs=127.17.54.254:36705
--webserver_port=32957
--webserver_interface=127.17.54.193
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.54.193
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:53:00.496584 27584 log.cc:824] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:53:00.498503 27584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:53:00.498821 27584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Bootstrap complete.
I20260812 03:53:00.501236 27584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } }
I20260812 03:53:00.501498 27584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0bdf61e65f425380feef307520a59a, State: Initialized, Role: FOLLOWER
I20260812 03:53:00.501690 27584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } }
I20260812 03:53:00.501770 27584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:53:00.501823 27584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:53:00.501880 27584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:53:00.503073 27584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } }
I20260812 03:53:00.503197 27584 leader_election.cc:302] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c0bdf61e65f425380feef307520a59a; no voters:
I20260812 03:53:00.503408 27584 leader_election.cc:288] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:53:00.503448 27588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:53:00.503580 27588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [term 2 LEADER]: Becoming Leader. State: Replica: 2c0bdf61e65f425380feef307520a59a, State: Running, Role: LEADER
I20260812 03:53:00.503700 27584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:53:00.503746 27588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } }
I20260812 03:53:00.504134 27588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c0bdf61e65f425380feef307520a59a. Latest consensus state: current_term: 2 leader_uuid: "2c0bdf61e65f425380feef307520a59a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } } }
I20260812 03:53:00.504132 27590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2c0bdf61e65f425380feef307520a59a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0bdf61e65f425380feef307520a59a" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36705 } } }
I20260812 03:53:00.504339 27588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: This master's current role is: LEADER
I20260812 03:53:00.504350 27590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a [sys.catalog]: This master's current role is: LEADER
I20260812 03:53:00.504616 27597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:53:00.505245 27597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:53:00.505656 27597 catalog_manager.cc:1259] Loaded cluster ID: f247bc9ae45d41478a654606072f0834
I20260812 03:53:00.505722 27597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:53:00.506626 27597 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:53:00.506814 27597 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2c0bdf61e65f425380feef307520a59a: Loaded TSK: 0
I20260812 03:53:00.507031 27597 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:53:00.590255 27586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:00.590516 27586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:00.590545 27586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:00.591845 27586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:00.591897 27586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:00.591928 27586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:53:00.593526 27586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.193.keytab
--principal=oryx/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:43751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=32957
--tserver_master_addrs=127.17.54.254:36705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:00.593822 27586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:00.594095 27586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:00.600494 27612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.600523 27616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.600503 27614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:00.600828 27586 server_base.cc:1034] running on GCE node
I20260812 03:53:00.601068 27586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:00.601365 27586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:00.602547 27586 hybrid_clock.cc:648] HybridClock initialized: now 1786506780602525 us; error 32 us; skew 500 ppm
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506780, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:00.605929 27586 init.cc:377] Logged in from keytab as oryx/127.17.54.193@KRBTEST.COM (short username oryx)
I20260812 03:53:00.606566 27586 webserver.cc:460] Webserver started at http://127.17.54.193:32957/ using document root <none> and password file <none>
I20260812 03:53:00.606844 27586 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:00.606945 27586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:00.608496 27586 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:53:00.609246 27623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.609493 27586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:00.609606 27586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "4ad8f7e5365b449cac42f3cfede010b3"
format_stamp: "Formatted at 2026-08-12 03:52:59 on dist-test-slave-tm1g"
server_key: "4499016303980b066df9fee9289e8e3f"
server_key_iv: "9ff4fe72011187f0e46e1c5977830a2b"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.610006 27586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:00.627580 27586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:00.630000 27586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:00.630208 27586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:00.630507 27586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:00.630879 27586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:00.630939 27586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.630985 27586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:00.631011 27586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.638639 27586 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:43751
I20260812 03:53:00.638672 27756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:43751 every 8 connection(s)
I20260812 03:53:00.639043 27586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:53:00.644691 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 27586
I20260812 03:53:00.645035 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:41315
--local_ip_for_outbound_sockets=127.17.54.194
--tserver_master_addrs=127.17.54.254:36705
--webserver_port=44283
--webserver_interface=127.17.54.194
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.54.194
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:00.649748 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.639879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:36361)
0812 03:53:00.640070 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.640117 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.640768 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.641252 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.641255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.641894 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.642100 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.642104 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.643873 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.643874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.644836 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.644839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.645724 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.645725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.645885 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.646576 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.646585 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.649499 (+ 2914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
W20260812 03:53:00.649801 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.640212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:36361 (local address 127.17.54.254:36705)
0812 03:53:00.640440 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.640456 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.640510 (+ 54us) server_negotiation.cc:407] Connection header received
0812 03:53:00.640992 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.640999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.641057 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.641208 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.642233 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.643738 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.644955 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.645605 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.645752 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.649637 (+ 3885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:36361: BlockingRecv error: recv got EOF from 127.17.54.193:36361 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
W20260812 03:53:00.650506 27757 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:36705 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.54.254:36705: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:00.655725 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.650816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:52777)
0812 03:53:00.650866 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.650887 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.650933 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.651091 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.651093 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.651245 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.651300 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.651301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.652017 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.652018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.652730 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.652733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.653138 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.653140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.653213 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.653707 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.653713 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.655631 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:53:00.655786 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.650857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:52777 (local address 127.17.54.254:36705)
0812 03:53:00.650903 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.650905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.650917 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:53:00.650990 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.650991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.651005 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.651088 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.651401 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.651930 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.652835 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.653047 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.653140 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.655715 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:52777: BlockingRecv error: recv got EOF from 127.17.54.193:52777 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:00.660998 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.656285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:36229)
0812 03:53:00.656379 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.656391 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.656444 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.656586 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.656587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.656728 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.656758 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.656759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.657436 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.657437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.658045 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.658047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.658523 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.658524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.658586 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.659078 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.659084 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.660904 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 03:53:00.661047 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.656338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:36229 (local address 127.17.54.254:36705)
0812 03:53:00.656390 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.656392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.656427 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:53:00.656508 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.656509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.656520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.656581 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.656847 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.657347 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.658135 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.658430 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.658523 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.660979 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:36229: BlockingRecv error: recv got EOF from 127.17.54.193:36229 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 03:53:00.661470 27757 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:53:00.734464 27761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:00.734735 27761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:00.734776 27761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:00.736006 27761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:00.736084 27761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:00.736129 27761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:53:00.737653 27761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.194.keytab
--principal=oryx/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:41315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=44283
--tserver_master_addrs=127.17.54.254:36705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:00.737953 27761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:00.738214 27761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:00.744103 27767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.744102 27766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.744102 27769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:00.744172 27761 server_base.cc:1034] running on GCE node
I20260812 03:53:00.744575 27761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:00.744836 27761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:00.746001 27761 hybrid_clock.cc:648] HybridClock initialized: now 1786506780745980 us; error 40 us; skew 500 ppm
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506780, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:00.748749 27761 init.cc:377] Logged in from keytab as oryx/127.17.54.194@KRBTEST.COM (short username oryx)
I20260812 03:53:00.749238 27761 webserver.cc:460] Webserver started at http://127.17.54.194:44283/ using document root <none> and password file <none>
I20260812 03:53:00.749481 27761 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:00.749577 27761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:00.751083 27761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:53:00.751830 27777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.752058 27761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:53:00.752169 27761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "f2eb5ca7f58c4e5d8b6af93fc48a4b83"
format_stamp: "Formatted at 2026-08-12 03:53:00 on dist-test-slave-tm1g"
server_key: "0d75d68b66455c93ec87580aeb179491"
server_key_iv: "e6bbc816c79398e4eed51eec5479164e"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.752552 27761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:00.761574 27761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:00.764081 27761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:00.764257 27761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:00.764523 27761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:00.764921 27761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:00.764986 27761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.765029 27761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:00.765057 27761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.771348 27761 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:41315
I20260812 03:53:00.771391 27914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:41315 every 8 connection(s)
I20260812 03:53:00.771750 27761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:53:00.772264 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 27761
I20260812 03:53:00.772598 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:33445
--local_ip_for_outbound_sockets=127.17.54.195
--tserver_master_addrs=127.17.54.254:36705
--webserver_port=43995
--webserver_interface=127.17.54.195
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.54.195
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:00.782389 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.774853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:57267)
0812 03:53:00.775036 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.775077 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.775628 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.775927 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.775931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.776651 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.776872 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.776884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.777689 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.777691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.778677 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.778679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.779099 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.779100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.779238 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.779880 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.779890 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.782153 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
W20260812 03:53:00.782423 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.774849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:57267 (local address 127.17.54.254:36705)
0812 03:53:00.774944 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.774946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.775172 (+ 226us) server_negotiation.cc:407] Connection header received
0812 03:53:00.775755 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.775758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.775778 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.775830 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.777001 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.777568 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.778775 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.779004 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.779089 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.782329 (+ 3240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:57267: BlockingRecv error: recv got EOF from 127.17.54.194:57267 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 03:53:00.782718 27915 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:36705 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.54.254:36705: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:00.787879 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.783029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:41861)
0812 03:53:00.783078 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.783088 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.783149 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.783303 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.783304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.783446 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.783493 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.783495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.784237 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.784238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.784829 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.784831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.785200 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.785201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.785270 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.785769 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.785775 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.787800 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:53:00.788025 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.783093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:41861 (local address 127.17.54.254:36705)
0812 03:53:00.783161 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.783163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.783173 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:00.783204 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.783205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.783215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.783290 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.783572 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.784135 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.784917 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.785126 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.785202 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.787909 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:41861: BlockingRecv error: recv got EOF from 127.17.54.194:41861 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:00.793103 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.788321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:59261)
0812 03:53:00.788380 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.788393 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.788477 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.788641 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.788642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.788785 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.788817 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.788819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.789523 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.789524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.790108 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.790110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.790575 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.790576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.790639 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.791106 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.791111 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.793023 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:53:00.793253 27915 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:53:00.793403 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.788408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:59261 (local address 127.17.54.254:36705)
0812 03:53:00.788463 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.788465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.788475 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:00.788548 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.788549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.788560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.788638 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.788909 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.789434 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.790204 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.790481 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.790573 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.793134 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:59261: BlockingRecv error: recv got EOF from 127.17.54.194:59261 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 03:53:00.858779 27917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:00.859018 27917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:00.859062 27917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:00.860292 27917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:00.860363 27917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:00.860407 27917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:53:00.861955 27917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/krb5kdc/oryx_127.17.54.195.keytab
--principal=oryx/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:33445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=43995
--tserver_master_addrs=127.17.54.254:36705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:00.862243 27917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:00.862536 27917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:00.868165 27926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:00.868134 27917 server_base.cc:1034] running on GCE node
W20260812 03:53:00.868156 27923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:00.868160 27924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:00.868574 27917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:00.868805 27917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:00.870014 27917 hybrid_clock.cc:648] HybridClock initialized: now 1786506780869998 us; error 31 us; skew 500 ppm
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506780, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:00.873005 27917 init.cc:377] Logged in from keytab as oryx/127.17.54.195@KRBTEST.COM (short username oryx)
I20260812 03:53:00.873534 27917 webserver.cc:460] Webserver started at http://127.17.54.195:43995/ using document root <none> and password file <none>
I20260812 03:53:00.873847 27917 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:00.873947 27917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:00.875479 27917 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:00.876353 27934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.876695 27917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:00.876852 27917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "6aa210951e4f4054acbc805621435936"
format_stamp: "Formatted at 2026-08-12 03:53:00 on dist-test-slave-tm1g"
server_key: "324bf8fa55dcc6f1cc6c0e8696aa3c50"
server_key_iv: "f57efcbd829683354d0f92dcce02c58d"
server_key_version: "encryptionkey@0"
I20260812 03:53:00.877336 27917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:00.888722 27917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:00.891569 27917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:00.891772 27917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:00.892611 27917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:00.892997 27917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:00.893082 27917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.893121 27917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:00.893167 27917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:00.899390 27917 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:33445
I20260812 03:53:00.899428 28066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:33445 every 8 connection(s)
I20260812 03:53:00.899803 27917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:53:00.908612 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.900648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:40001 (local address 127.17.54.254:36705)
0812 03:53:00.900702 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.900705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.900934 (+ 229us) server_negotiation.cc:407] Connection header received
0812 03:53:00.901502 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.901503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.901518 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.901566 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.902658 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.903247 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.904497 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.904696 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.904784 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.908561 (+ 3777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:40001: BlockingRecv error: recv got EOF from 127.17.54.195:40001 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:53:00.908617 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.900577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:40001)
0812 03:53:00.900757 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.900836 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.901383 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.901640 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.901644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.902258 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.902547 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.902553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.903398 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.903399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.904403 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.904406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.904815 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.904817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.904953 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.905570 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.905580 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.908456 (+ 2876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
W20260812 03:53:00.908922 28068 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:36705 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.54.254:36705: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
I20260812 03:53:00.909053 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 27917
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:53:00.914466 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.909365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:54751 (local address 127.17.54.254:36705)
0812 03:53:00.909406 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.909408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.909415 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:53:00.909442 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.909442 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.909453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.909516 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.909828 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.910451 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.911168 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.911343 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.911399 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.914292 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:54751: BlockingRecv error: recv got EOF from 127.17.54.195:54751 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:53:00.914654 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.909193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:54751)
0812 03:53:00.909253 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.909267 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.909318 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.909522 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.909524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.909691 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.909755 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.909757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.910537 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.910539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.911088 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.911090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.911414 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.911416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.911482 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.912174 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.912180 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.914573 (+ 2393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 03:53:00.917906 28071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.909439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.17.54.254:36705)
0812 03:53:00.909545 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.909546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.909718 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:53:00.909755 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.909756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.909765 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.909847 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.910470 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.910951 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.911935 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.912358 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.912421 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.913865 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:00.913871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:00.913871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:53:00.913939 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:00.916228 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:00.916688 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:00.916692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:00.916692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:00.916802 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:00.917015 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:00.917016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:00.917016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:00.917344 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:00.917507 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:00.917658 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 03:53:00.917788 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":57,"threads_started":1}
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:00 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:00.919911 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:00.915023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:33199)
0812 03:53:00.915124 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:00.915135 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:00.915185 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:00.915326 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:00.915327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:00.915441 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:00.915468 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.915469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.916315 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.916316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.917031 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:00.917034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.917452 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.917453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:00.917505 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.917897 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:00.917901 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:00.919817 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260812 03:53:00.919942 27760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:00.915079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:33199 (local address 127.17.54.254:36705)
0812 03:53:00.915115 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:00.915117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:00.915169 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:53:00.915241 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:00.915242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:00.915251 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:00.915306 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:00.915545 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.916231 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.917113 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:00.917385 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:00.917446 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:00.919882 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:33199: BlockingRecv error: recv got EOF from 127.17.54.195:33199 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 03:53:00.920042 28068 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 03:53:01 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:01 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:01.667358 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:01.661914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:34325)
0812 03:53:01.662090 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:01.662102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:01.662156 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:01.662500 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:01.662501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:01.662690 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:01.662753 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:01.662755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.663589 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.663590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:01.664168 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:01.664169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.664614 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.664615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:01.664668 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:01.665265 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:01.665270 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:01.667238 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
W20260812 03:53:01.667512 28074 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:01.661998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:34325 (local address 127.17.54.254:36705)
0812 03:53:01.662156 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:01.662158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:01.662167 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:53:01.662209 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:01.662210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:01.662220 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:01.662290 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:01.662910 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.663492 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.664254 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.664509 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.664569 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:01.667440 (+ 2871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:34325: BlockingRecv error: recv got EOF from 127.17.54.193:34325 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
Aug 12 03:53:01 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:01 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:01.798779 28075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:01.793750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:60403)
0812 03:53:01.793977 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:01.793990 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:01.794088 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:01.794237 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:01.794238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:01.794475 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:01.794527 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:01.794529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.795250 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.795252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:01.795812 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:01.795813 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.796185 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.796185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:01.796245 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:01.796755 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:01.796765 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:01.798705 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":122,"threads_started":1}
W20260812 03:53:01.798914 28074 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:01.793855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:60403 (local address 127.17.54.254:36705)
0812 03:53:01.793964 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:01.793968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:01.794068 (+ 100us) server_negotiation.cc:407] Connection header received
0812 03:53:01.794157 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:01.794158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:01.794172 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:01.794222 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:01.794614 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.795153 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.795903 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.796110 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.796187 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:01.798830 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:60403: BlockingRecv error: recv got EOF from 127.17.54.194:60403 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 03:53:01 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:01 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:01.925570 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:01.920482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:50317)
0812 03:53:01.920665 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:01.920677 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:01.920746 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:01.920881 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:01.920882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:01.921054 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:01.921107 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:01.921109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.921826 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.921827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:01.922739 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:01.922742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.923088 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.923089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:01.923132 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:01.923590 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:01.923595 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:01.925477 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
W20260812 03:53:01.925629 28074 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:01.920595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:50317 (local address 127.17.54.254:36705)
0812 03:53:01.920669 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:01.920671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:01.920729 (+ 58us) server_negotiation.cc:407] Connection header received
0812 03:53:01.920803 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:01.920804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:01.920817 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:01.920857 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:01.921173 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.921716 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.922823 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:01.923019 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:01.923075 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:01.925581 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:50317: BlockingRecv error: recv got EOF from 127.17.54.195:50317 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 03:53:02 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:02 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:02.673223 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:02.668036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:53437)
0812 03:53:02.668169 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:02.668181 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:02.668239 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:02.668452 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:02.668453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:02.668624 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:02.668684 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:02.668686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.669630 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.669631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:02.670169 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:02.670171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.670606 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.670607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:02.670655 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:02.671147 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:02.671153 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:02.673161 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
W20260812 03:53:02.673374 28078 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:02.668135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:53437 (local address 127.17.54.254:36705)
0812 03:53:02.668278 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:02.668281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:02.668289 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:53:02.668323 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:02.668323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:02.668336 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:02.668408 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:02.668787 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.669563 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.670247 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.670527 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.670584 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:02.673290 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:53437: BlockingRecv error: recv got EOF from 127.17.54.193:53437 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
Aug 12 03:53:02 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:02 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:02.805011 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:02.799337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:35725)
0812 03:53:02.799502 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:02.799515 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:02.799594 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:02.799770 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:02.799772 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:02.799944 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:02.800000 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:02.800002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.800677 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.800677 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:02.801462 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:02.801469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.802158 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.802159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:02.802210 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:02.802785 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:02.802791 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:02.804947 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":139,"threads_started":1}
W20260812 03:53:02.805194 28078 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:02.799446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:35725 (local address 127.17.54.254:36705)
0812 03:53:02.799519 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:02.799521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:02.799585 (+ 64us) server_negotiation.cc:407] Connection header received
0812 03:53:02.799645 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:02.799646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:02.799658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:02.799721 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:02.800075 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.800597 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.801523 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.802029 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.802104 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:02.805123 (+ 3019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:35725: BlockingRecv error: recv got EOF from 127.17.54.194:35725 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 03:53:02 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:02 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:02.931242 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:02.926147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:33833)
0812 03:53:02.926340 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:02.926350 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:02.926401 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:02.926597 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:02.926598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:02.926773 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:02.926828 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:02.926829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.927570 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.927571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:02.928178 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:02.928180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.928605 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.928607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:02.928658 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:02.929227 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:02.929232 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:02.931178 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
W20260812 03:53:02.931545 28078 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:02.926389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:33833 (local address 127.17.54.254:36705)
0812 03:53:02.926469 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:02.926471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:02.926480 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:53:02.926502 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:02.926503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:02.926515 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:02.926576 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:02.926955 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.927457 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.928275 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:02.928490 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:02.928550 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:02.931437 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:33833: BlockingRecv error: recv got EOF from 127.17.54.195:33833 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 03:53:03 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:03 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:03.684588 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:03.673785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:47709)
0812 03:53:03.673968 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:03.673979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:03.674037 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:03.674220 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:03.674221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:03.674472 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:03.674531 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:03.674532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.675265 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.675266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:03.675806 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:03.675807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.676204 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.676205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:03.676258 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:03.682382 (+ 6124us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:03.682390 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:03.684518 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
W20260812 03:53:03.684803 28082 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:03.673858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:47709 (local address 127.17.54.254:36705)
0812 03:53:03.674042 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:03.674044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:03.674061 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:53:03.674107 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:03.674107 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:03.674118 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:03.674178 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:03.674647 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.675176 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.675919 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.676122 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.676188 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:03.684720 (+ 8532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:47709: BlockingRecv error: recv got EOF from 127.17.54.193:47709 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
Aug 12 03:53:03 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:03 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:03.811074 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:03.805543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:56817)
0812 03:53:03.805687 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:03.805707 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:03.805775 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:03.805940 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:03.805941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:03.806160 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:03.806241 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:03.806243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.807077 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.807078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:03.807680 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:03.807682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.808095 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.808096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:03.808176 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:03.808701 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:03.808706 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:03.811000 (+ 2294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
W20260812 03:53:03.811228 28082 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:03.805590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:56817 (local address 127.17.54.254:36705)
0812 03:53:03.805660 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:03.805663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:03.805818 (+ 155us) server_negotiation.cc:407] Connection header received
0812 03:53:03.805852 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:03.805853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:03.805868 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:03.805930 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:03.806394 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.806975 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.807776 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.808013 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.808097 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:03.811146 (+ 3049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:56817: BlockingRecv error: recv got EOF from 127.17.54.194:56817 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 03:53:03 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:03 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:03.937292 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:03.931803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:39327)
0812 03:53:03.932168 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:03.932183 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:03.932271 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:03.932465 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:03.932465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:03.932685 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:03.932741 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:03.932743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.933529 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.933530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:03.934125 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:03.934127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.934621 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.934622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:03.934701 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:03.935203 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:03.935208 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:03.937224 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":125,"threads_started":1}
W20260812 03:53:03.937405 28082 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:03.931959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:39327 (local address 127.17.54.254:36705)
0812 03:53:03.932155 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:03.932158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:03.932266 (+ 108us) server_negotiation.cc:407] Connection header received
0812 03:53:03.932347 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:03.932349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:03.932372 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:03.932453 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:03.932852 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.933422 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.934229 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:03.934522 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:03.934607 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:03.937330 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:39327: BlockingRecv error: recv got EOF from 127.17.54.195:39327 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Aug 12 03:53:04 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:04 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:04.690465 28086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:04.685235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:51649)
0812 03:53:04.685427 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:04.685439 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:04.685496 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:04.685703 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:04.685704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:04.686020 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:04.686092 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:04.686095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.686950 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.686951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:04.687485 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:04.687487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.687830 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.687831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:04.687877 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:04.688402 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:04.688406 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:04.690398 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":119,"threads_started":1}
W20260812 03:53:04.690629 28087 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:04.685374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:51649 (local address 127.17.54.254:36705)
0812 03:53:04.685547 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:04.685550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:04.685560 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:04.685584 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:04.685585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:04.685596 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:04.685667 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:04.686187 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.686857 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.687572 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.687763 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.687831 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:04.690526 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:51649: BlockingRecv error: recv got EOF from 127.17.54.193:51649 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
Aug 12 03:53:04 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:04 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:04.816642 28088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:04.811652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:43045)
0812 03:53:04.811831 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:04.811843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:04.811922 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:04.812077 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:04.812078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:04.812253 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:04.812319 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:04.812321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.813051 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.813051 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:04.813651 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:04.813653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.814060 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.814061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:04.814147 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:04.814689 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:04.814695 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:04.816578 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
W20260812 03:53:04.817008 28087 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:04.811763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:43045 (local address 127.17.54.254:36705)
0812 03:53:04.811823 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:04.811826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:04.811927 (+ 101us) server_negotiation.cc:407] Connection header received
0812 03:53:04.811999 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:04.812000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:04.812013 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:04.812059 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:04.812403 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.812975 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.813731 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.813951 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.814037 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:04.816923 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:43045: BlockingRecv error: recv got EOF from 127.17.54.194:43045 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 03:53:04 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:04 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:04.942978 28089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:04.938031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:43643)
0812 03:53:04.938160 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:04.938172 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:04.938239 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:04.938413 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:04.938414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:04.938586 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:04.938638 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:04.938640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.939393 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.939395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:04.939944 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:04.939946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.940346 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.940347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:04.940417 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:04.940943 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:04.940948 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:04.942876 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
W20260812 03:53:04.943027 28087 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:04.938111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:43643 (local address 127.17.54.254:36705)
0812 03:53:04.938162 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:04.938165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:04.938228 (+ 63us) server_negotiation.cc:407] Connection header received
0812 03:53:04.938284 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:04.938284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:04.938325 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:04.938410 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:04.938748 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.939265 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.940053 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:04.940261 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:04.940347 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:04.942960 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:43643: BlockingRecv error: recv got EOF from 127.17.54.195:43643 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 03:53:05 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:05 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:05.696312 28090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:05.691104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:36325)
0812 03:53:05.691280 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:05.691296 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:05.691357 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:05.691614 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:05.691615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:05.691792 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:05.691843 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:05.691845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.692585 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.692586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:05.693112 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:05.693114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.693475 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.693476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:05.693536 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:05.694256 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:05.694261 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:05.696250 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
W20260812 03:53:05.696458 28091 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:05.691212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:36325 (local address 127.17.54.254:36705)
0812 03:53:05.691345 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:05.691347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:05.691358 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:53:05.691445 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:05.691446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:05.691457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:05.691552 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:05.691940 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.692510 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.693196 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.693392 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.693526 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:05.696361 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:36325: BlockingRecv error: recv got EOF from 127.17.54.193:36325 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
Aug 12 03:53:05 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:05 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:05.822505 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:05.817157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:37609)
0812 03:53:05.817499 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:05.817512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:05.817604 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:05.817765 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:05.817766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:05.817963 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:05.818019 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:05.818021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.818776 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.818778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:05.819312 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:05.819314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.819678 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.819679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:05.819737 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:05.820246 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:05.820251 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:05.822406 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":115,"threads_started":1}
W20260812 03:53:05.822542 28091 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:05.817288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:37609 (local address 127.17.54.254:36705)
0812 03:53:05.817465 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:05.817467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:05.817582 (+ 115us) server_negotiation.cc:407] Connection header received
0812 03:53:05.817669 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:05.817670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:05.817683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:05.817758 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:05.818101 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.818686 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.819400 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.819600 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.819680 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:05.822479 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:37609: BlockingRecv error: recv got EOF from 127.17.54.194:37609 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Aug 12 03:53:05 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:05 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:05.948979 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:05.943555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:54635)
0812 03:53:05.943755 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:05.943767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:05.943871 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:05.944041 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:05.944042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:05.944237 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:05.944288 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:05.944289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.945008 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.945009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:05.945561 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:05.945562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.945930 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.945931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:05.946002 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:05.946661 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:05.946667 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:05.948877 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
W20260812 03:53:05.949038 28091 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:05.943657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:54635 (local address 127.17.54.254:36705)
0812 03:53:05.943735 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:05.943738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:05.943849 (+ 111us) server_negotiation.cc:407] Connection header received
0812 03:53:05.943931 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:05.943932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:05.943949 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:05.944021 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:05.944379 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.944913 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.945654 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:05.945851 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:05.945931 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:05.948973 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:54635: BlockingRecv error: recv got EOF from 127.17.54.195:54635 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 03:53:06 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:06 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:06.702466 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:06.696949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:41813)
0812 03:53:06.697145 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:06.697162 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:06.697232 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:06.697377 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:06.697378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:06.697550 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:06.697605 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:06.697606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.698608 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.698609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:06.699169 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:06.699171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.699694 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.699695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:06.699772 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:06.700343 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:06.700354 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:06.702356 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":143,"threads_started":1}
W20260812 03:53:06.702523 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:06.697091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:41813 (local address 127.17.54.254:36705)
0812 03:53:06.697227 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:06.697229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:06.697240 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:53:06.697285 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:06.697285 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:06.697296 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:06.697369 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:06.697740 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.698499 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.699290 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.699600 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.699693 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:06.702444 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:41813: BlockingRecv error: recv got EOF from 127.17.54.193:41813 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":58,"threads_started":1}
Aug 12 03:53:06 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:06 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:06.827975 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:06.823070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:51957)
0812 03:53:06.823211 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:06.823229 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:06.823297 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:06.823460 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:06.823461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:06.823635 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:06.823688 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:06.823689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.824513 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.824514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:06.825074 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:06.825076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.825524 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.825525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:06.825575 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:06.826054 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:06.826060 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:06.827895 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
W20260812 03:53:06.828132 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:06.823133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:51957 (local address 127.17.54.254:36705)
0812 03:53:06.823215 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:06.823217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:06.823343 (+ 126us) server_negotiation.cc:407] Connection header received
0812 03:53:06.823377 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:06.823378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:06.823391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:06.823451 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:06.823804 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.824420 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.825172 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.825442 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.825507 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:06.828047 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:51957: BlockingRecv error: recv got EOF from 127.17.54.194:51957 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 03:53:06 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:06 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:06.954399 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:06.949556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:34613)
0812 03:53:06.949718 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:06.949734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:06.949791 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:06.949934 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:06.949935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:06.950104 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:06.950158 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:06.950160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.950919 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.950920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:06.951476 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:06.951478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.951826 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.951827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:06.951871 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:06.952370 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:06.952374 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:06.954250 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
W20260812 03:53:06.954474 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:06.949666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:34613 (local address 127.17.54.254:36705)
0812 03:53:06.949718 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:06.949720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:06.949780 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:53:06.949854 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:06.949855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:06.949867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:06.949922 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:06.950260 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.950806 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.951569 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:06.951754 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:06.951812 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:06.954424 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:34613: BlockingRecv error: recv got EOF from 127.17.54.195:34613 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 03:53:07 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:07 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:07.708407 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:07.703093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:55701)
0812 03:53:07.703254 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:07.703266 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:07.703329 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:07.703520 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:07.703521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:07.703792 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:07.703847 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:07.703849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.704618 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.704619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:07.705263 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:07.705266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.705730 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.705731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:07.705779 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:07.706274 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:07.706279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:07.708334 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
W20260812 03:53:07.708619 28101 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:07.703227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:55701 (local address 127.17.54.254:36705)
0812 03:53:07.703405 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:07.703407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:07.703417 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:07.703442 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:07.703442 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:07.703453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:07.703512 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:07.703928 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.704489 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.705419 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.705650 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.705722 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:07.708468 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:55701: BlockingRecv error: recv got EOF from 127.17.54.193:55701 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
Aug 12 03:53:07 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:07 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:07.833106 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:07.828575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:52121)
0812 03:53:07.828744 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:07.828754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:07.828808 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:07.828920 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:07.828921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:07.829106 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:07.829159 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:07.829160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.829807 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.829808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:07.830385 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:07.830387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.830725 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.830726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:07.830775 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:07.831229 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:07.831234 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:07.833034 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
W20260812 03:53:07.833297 28101 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:07.828653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:52121 (local address 127.17.54.254:36705)
0812 03:53:07.828720 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:07.828722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:07.828826 (+ 104us) server_negotiation.cc:407] Connection header received
0812 03:53:07.828849 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:07.828850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:07.828862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:07.828919 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:07.829218 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.829732 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.830457 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.830649 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.830708 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:07.833226 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:52121: BlockingRecv error: recv got EOF from 127.17.54.194:52121 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 03:53:07 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:07 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:07.960201 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:07.955042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:57229)
0812 03:53:07.955255 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:07.955267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:07.955358 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:07.955598 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:07.955599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:07.955769 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:07.955827 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:07.955829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.956617 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.956618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:07.957155 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:07.957157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.957527 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.957528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:07.957576 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:07.958091 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:07.958096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:07.960094 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":132,"threads_started":1}
W20260812 03:53:07.960304 28101 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:07.955152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:57229 (local address 127.17.54.254:36705)
0812 03:53:07.955237 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:07.955240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:07.955337 (+ 97us) server_negotiation.cc:407] Connection header received
0812 03:53:07.955475 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:07.955476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:07.955498 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:07.955576 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:07.955926 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.956493 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.957238 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:07.957454 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:07.957512 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:07.960208 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:57229: BlockingRecv error: recv got EOF from 127.17.54.195:57229 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 03:53:08 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:08 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:53:08.718668 28106 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:08.709177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:36891 (local address 127.17.54.254:36705)
0812 03:53:08.709358 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:08.709360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:08.709370 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:08.709423 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:08.709424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:08.709436 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:08.709507 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:08.709900 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.710474 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.711359 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.711585 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.711649 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:08.718566 (+ 6917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:36891: BlockingRecv error: recv got EOF from 127.17.54.193:36891 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260812 03:53:08.718871 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:08.709014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:36891)
0812 03:53:08.709232 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:08.709244 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:08.709305 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:08.709560 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:08.709561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:08.709741 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:08.709801 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:08.709802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.710641 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.710642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:08.711219 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:08.711222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.711675 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.711676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:08.711730 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:08.716007 (+ 4277us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:08.716015 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:08.718793 (+ 2778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":122,"threads_started":1}
Aug 12 03:53:08 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:08 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:08.840417 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:08.833648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:40573)
0812 03:53:08.833842 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:08.833856 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:08.833946 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:08.834132 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:08.834133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:08.834376 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:08.834465 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:08.834468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.835745 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.835747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:08.836697 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:08.836809 (+ 112us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.837384 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.837385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:08.837439 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:08.838021 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:08.838027 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:08.840314 (+ 2287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":74,"threads_started":1}
W20260812 03:53:08.840485 28106 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:08.833724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:40573 (local address 127.17.54.254:36705)
0812 03:53:08.833803 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:08.833805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:08.833918 (+ 113us) server_negotiation.cc:407] Connection header received
0812 03:53:08.834028 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:08.834029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:08.834045 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:08.834112 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:08.834557 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.835631 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.836923 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.837256 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.837328 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:08.840421 (+ 3093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:40573: BlockingRecv error: recv got EOF from 127.17.54.194:40573 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 03:53:08 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:08 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:08.966357 28108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:08.960862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:56043)
0812 03:53:08.961000 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:08.961012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:08.961078 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:08.961274 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:08.961275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:08.961514 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:08.961577 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:08.961580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.962531 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.962532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:08.963068 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:08.963070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.963462 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.963463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:08.963520 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:08.964047 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:08.964052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:08.966228 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":94,"threads_started":1}
W20260812 03:53:08.966619 28106 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:08.961041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:56043 (local address 127.17.54.254:36705)
0812 03:53:08.961114 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:08.961116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:08.961126 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:08.961152 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:08.961153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:08.961166 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:08.961261 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:08.961735 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.962452 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.963155 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:08.963350 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:08.963425 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:08.966480 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:56043: BlockingRecv error: recv got EOF from 127.17.54.195:56043 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:53:09 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:09 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:09.724466 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:09.719468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:53327)
0812 03:53:09.719619 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:09.719630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:09.719679 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:09.719885 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:09.719885 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:09.720064 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:09.720119 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:09.720120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.720828 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.720829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:09.721366 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:09.721368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.721753 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.721754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:09.721799 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:09.722297 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:09.722354 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:09.724389 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
W20260812 03:53:09.724654 28110 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:09.719605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:53327 (local address 127.17.54.254:36705)
0812 03:53:09.719743 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:09.719745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:09.719762 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:53:09.719786 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:09.719787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:09.719800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:09.719870 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:09.720188 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.720756 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.721464 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.721674 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.721749 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:09.724541 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:53327: BlockingRecv error: recv got EOF from 127.17.54.193:53327 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
Aug 12 03:53:09 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:09 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:09.845666 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:09.840950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:47207)
0812 03:53:09.841113 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:09.841124 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:09.841187 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:09.841334 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:09.841334 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:09.841516 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:09.841568 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:09.841570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.842233 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.842234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:09.842817 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:09.842819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.843173 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.843173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:09.843216 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:09.843681 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:09.843686 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:09.845591 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
W20260812 03:53:09.845806 28110 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:09.841029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:47207 (local address 127.17.54.254:36705)
0812 03:53:09.841079 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:09.841081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:09.841171 (+ 90us) server_negotiation.cc:407] Connection header received
0812 03:53:09.841251 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:09.841252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:09.841262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:09.841308 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:09.841639 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.842157 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.842909 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.843103 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.843159 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:09.845723 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:47207: BlockingRecv error: recv got EOF from 127.17.54.194:47207 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 03:53:09 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:09 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:09.972091 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:09.966972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:50201)
0812 03:53:09.967156 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:09.967177 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:09.967252 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:09.967404 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:09.967405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:09.967587 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:09.967647 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:09.967648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.968378 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.968380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:09.968958 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:09.968960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.969441 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.969443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:09.969559 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:09.970030 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:09.970036 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:09.971989 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":74,"threads_started":1}
W20260812 03:53:09.972137 28110 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:09.967094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:50201 (local address 127.17.54.254:36705)
0812 03:53:09.967155 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:09.967157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:09.967248 (+ 91us) server_negotiation.cc:407] Connection header received
0812 03:53:09.967302 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:09.967303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:09.967320 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:09.967386 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:09.967720 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.968269 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.969036 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:09.969364 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:09.969427 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:09.972072 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:50201: BlockingRecv error: recv got EOF from 127.17.54.195:50201 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 03:53:10 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:10 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:10.730522 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:10.725135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:58265)
0812 03:53:10.725336 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:10.725350 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:10.725424 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:10.725647 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:10.725648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:10.725828 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:10.725887 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:10.725888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.726666 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.726667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:10.727232 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:10.727233 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.727662 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.727663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:10.727720 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:10.728334 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:10.728340 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:10.730440 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
W20260812 03:53:10.730598 28114 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:10.725249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:58265 (local address 127.17.54.254:36705)
0812 03:53:10.725472 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:10.725475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:10.725485 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:10.725510 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:10.725511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:10.725523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:10.725599 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:10.725970 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.726570 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.727343 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.727567 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.727631 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:10.730535 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:58265: BlockingRecv error: recv got EOF from 127.17.54.193:58265 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":116,"threads_started":1}
Aug 12 03:53:10 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:10 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:10.851083 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:10.846221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:58307)
0812 03:53:10.846408 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:10.846420 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:10.846490 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:10.846665 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:10.846666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:10.846837 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:10.846892 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:10.846894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.847588 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.847589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:10.848136 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:10.848137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.848560 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.848560 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:10.848604 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:10.849125 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:10.849131 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:10.851022 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":129,"threads_started":1}
W20260812 03:53:10.851365 28114 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:10.846286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:58307 (local address 127.17.54.254:36705)
0812 03:53:10.846413 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:10.846415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:10.846488 (+ 73us) server_negotiation.cc:407] Connection header received
0812 03:53:10.846552 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:10.846552 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:10.846565 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:10.846642 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:10.846981 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.847513 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.848263 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.848483 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.848548 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:10.851295 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:58307: BlockingRecv error: recv got EOF from 127.17.54.194:58307 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 12 03:53:10 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:10 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:10.977654 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:10.972651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:44515)
0812 03:53:10.972834 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:10.972848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:10.972915 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:10.973105 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:10.973106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:10.973295 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:10.973353 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:10.973354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.974049 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.974050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:10.974629 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:10.974631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.975018 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.975019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:10.975087 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:10.975552 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:10.975557 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:10.977538 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":120,"threads_started":1}
W20260812 03:53:10.977687 28114 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:10.972798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:44515 (local address 127.17.54.254:36705)
0812 03:53:10.972868 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:10.972870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:10.972897 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:53:10.973002 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:10.973003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:10.973020 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:10.973094 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:10.973441 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.973959 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.974729 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:10.974933 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:10.975013 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:10.977637 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:44515: BlockingRecv error: recv got EOF from 127.17.54.195:44515 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 03:53:11 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:11 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:11.736259 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:11.731250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:58773)
0812 03:53:11.731435 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:11.731447 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:11.731512 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:11.731805 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:11.731806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:11.732097 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:11.732165 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:11.732168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.732874 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.732875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:11.733425 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:11.733426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.733775 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.733776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:11.733818 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:11.734282 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:11.734287 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:11.736185 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":123,"threads_started":1}
W20260812 03:53:11.736361 28118 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:11.731472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:58773 (local address 127.17.54.254:36705)
0812 03:53:11.731606 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:11.731608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:11.731647 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:53:11.731679 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:11.731680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:11.731693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:11.731787 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:11.732262 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.732792 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.733508 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.733707 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.733763 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:11.736286 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:58773: BlockingRecv error: recv got EOF from 127.17.54.193:58773 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
Aug 12 03:53:11 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:11 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:11.857080 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:11.851715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:51879)
0812 03:53:11.852026 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:11.852040 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:11.852122 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:11.852285 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:11.852286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:11.852496 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:11.852545 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:11.852547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.853302 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.853303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:11.853852 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:11.853854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.854207 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.854208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:11.854255 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:11.854850 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:11.854857 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:11.856944 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":207,"threads_started":1}
W20260812 03:53:11.857113 28118 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:11.851928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:51879 (local address 127.17.54.254:36705)
0812 03:53:11.852012 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:11.852015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:11.852100 (+ 85us) server_negotiation.cc:407] Connection header received
0812 03:53:11.852187 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:11.852188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:11.852208 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:11.852277 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:11.852657 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.853185 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.853932 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.854127 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.854186 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:11.857045 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:51879: BlockingRecv error: recv got EOF from 127.17.54.194:51879 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 03:53:11 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:11 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:11.983501 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:11.978425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:33483)
0812 03:53:11.978573 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:11.978586 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:11.978686 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:11.978839 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:11.978839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:11.979025 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:11.979100 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:11.979102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.979864 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.979866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:11.980496 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:11.980499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.980856 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.980857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:11.980930 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:11.981420 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:11.981426 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:11.983386 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
W20260812 03:53:11.983551 28118 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:11.978502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:33483 (local address 127.17.54.254:36705)
0812 03:53:11.978570 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:11.978572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:11.978711 (+ 139us) server_negotiation.cc:407] Connection header received
0812 03:53:11.978744 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:11.978745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:11.978758 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:11.978821 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:11.979172 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.979718 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.980591 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:11.980781 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:11.980839 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:11.983471 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:33483: BlockingRecv error: recv got EOF from 127.17.54.195:33483 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:53:12 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:12 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:12.742033 28121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:12.736893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:36831)
0812 03:53:12.737067 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:12.737080 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:12.737167 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:12.737353 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:12.737354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:12.737570 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:12.737633 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:12.737635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.738391 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.738392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:12.738923 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:12.738924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.739279 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.739280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:12.739323 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:12.739886 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:12.739890 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:12.741968 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
W20260812 03:53:12.742204 28122 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:12.736934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:36831 (local address 127.17.54.254:36705)
0812 03:53:12.737063 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:12.737065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:12.737161 (+ 96us) server_negotiation.cc:407] Connection header received
0812 03:53:12.737225 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:12.737226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:12.737239 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:12.737338 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:12.737755 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.738291 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.738999 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.739205 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.739262 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:12.742097 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:36831: BlockingRecv error: recv got EOF from 127.17.54.193:36831 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
Aug 12 03:53:12 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:12 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:12.862926 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:12.857769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:55497)
0812 03:53:12.857928 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:12.857948 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:12.858034 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:12.858244 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:12.858246 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:12.858555 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:12.858622 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:12.858624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.859354 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.859355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:12.859918 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:12.859920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.860313 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.860314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:12.860375 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:12.860908 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:12.860914 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:12.862822 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
W20260812 03:53:12.862991 28122 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:12.857977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:55497 (local address 127.17.54.254:36705)
0812 03:53:12.858043 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:12.858045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:12.858111 (+ 66us) server_negotiation.cc:407] Connection header received
0812 03:53:12.858140 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:12.858141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:12.858154 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:12.858229 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:12.858709 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.859255 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.860009 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.860235 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.860315 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:12.862916 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:55497: BlockingRecv error: recv got EOF from 127.17.54.194:55497 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 03:53:12 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:12 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:12.989250 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:12.984067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:37207)
0812 03:53:12.984269 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:12.984281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:12.984349 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:12.984492 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:12.984493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:12.984685 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:12.984743 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:12.984744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.985556 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.985557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:12.986150 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:12.986151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.986614 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.986615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:12.986676 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:12.987186 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:12.987192 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:12.989132 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":105,"threads_started":1}
W20260812 03:53:12.989285 28122 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:12.984136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:37207 (local address 127.17.54.254:36705)
0812 03:53:12.984211 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:12.984214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:12.984354 (+ 140us) server_negotiation.cc:407] Connection header received
0812 03:53:12.984395 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:12.984396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:12.984409 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:12.984461 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:12.984869 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.985474 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.986261 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:12.986515 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:12.986575 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:12.989217 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:37207: BlockingRecv error: recv got EOF from 127.17.54.195:37207 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 03:53:13 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:13 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:13.748629 28126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:13.742716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:44491)
0812 03:53:13.742867 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:13.742878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:13.742962 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:13.743275 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:13.743276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:13.743466 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:13.743521 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:13.743523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.744248 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.744249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:13.744810 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:13.744811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.745276 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.745277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:13.745361 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:13.745867 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:13.745874 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:13.748514 (+ 2640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
W20260812 03:53:13.748678 28127 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:13.742787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:44491 (local address 127.17.54.254:36705)
0812 03:53:13.743102 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:13.743104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:13.743131 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:53:13.743165 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:13.743166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:13.743178 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:13.743239 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:13.743623 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.744156 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.744901 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.745179 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.745576 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:13.748600 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:44491: BlockingRecv error: recv got EOF from 127.17.54.193:44491 (error 108)
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":85,"threads_started":1}
Aug 12 03:53:13 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:13 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:13.868753 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:13.863521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:36313)
0812 03:53:13.863680 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:13.863691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:13.863748 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:13.863957 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:13.863967 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:13.864150 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:13.864201 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:13.864203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.864974 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.864976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:13.865567 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:13.865570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.865983 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.865984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:13.866031 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:13.866681 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:13.866686 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:13.868683 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
W20260812 03:53:13.868870 28127 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:13.863715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:36313 (local address 127.17.54.254:36705)
0812 03:53:13.863782 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:13.863788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:13.863796 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:53:13.863823 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:13.863824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:13.863839 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:13.863898 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:13.864342 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.864866 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.865680 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.865895 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.865961 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:13.868793 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:36313: BlockingRecv error: recv got EOF from 127.17.54.194:36313 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 03:53:13 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:13 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:13.995251 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:13.989839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:50773)
0812 03:53:13.989984 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:13.989996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:13.990060 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:13.990224 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:13.990224 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:13.990425 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:13.990483 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:13.990484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.991179 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.991180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:13.991722 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:13.991724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.992176 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.992177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:13.992232 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:13.992778 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:13.992783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:13.995188 (+ 2405us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
W20260812 03:53:13.995443 28127 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:13.990001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:50773 (local address 127.17.54.254:36705)
0812 03:53:13.990083 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:13.990090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:13.990100 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:13.990126 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:13.990126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:13.990141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:13.990207 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:13.990573 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.991103 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.991895 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:13.992086 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:13.992151 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:13.995381 (+ 3230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:50773: BlockingRecv error: recv got EOF from 127.17.54.195:50773 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 03:53:14 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:14 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:14.754451 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:14.749305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:47661)
0812 03:53:14.749510 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:14.749520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:14.749574 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:14.749753 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:14.749754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:14.749936 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:14.749991 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:14.749992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.750747 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.750749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:14.751291 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:14.751293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.751689 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.751691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:14.751743 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:14.752237 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:14.752242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:14.754385 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
W20260812 03:53:14.754616 28131 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:14.749449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:47661 (local address 127.17.54.254:36705)
0812 03:53:14.749600 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:14.749602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:14.749612 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:14.749647 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:14.749648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:14.749659 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:14.749730 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:14.750075 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.750647 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.751404 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.751606 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.751662 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:14.754531 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:47661: BlockingRecv error: recv got EOF from 127.17.54.193:47661 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
Aug 12 03:53:14 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:14 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:14.874469 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:14.869365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:54011)
0812 03:53:14.869615 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:14.869630 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:14.869714 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:14.869871 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:14.869872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:14.870044 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:14.870098 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:14.870099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.870843 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.870844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:14.871419 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:14.871421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.871832 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.871833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:14.871884 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:14.872400 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:14.872405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:14.874366 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":113,"threads_started":1}
W20260812 03:53:14.874528 28131 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:14.869428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:54011 (local address 127.17.54.254:36705)
0812 03:53:14.869505 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:14.869508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:14.869733 (+ 225us) server_negotiation.cc:407] Connection header received
0812 03:53:14.869773 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:14.869774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:14.869786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:14.869849 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:14.870169 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.870738 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.871510 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.871728 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.871789 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:14.874465 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:54011: BlockingRecv error: recv got EOF from 127.17.54.194:54011 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 03:53:14 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:14 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.195@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:15.001433 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:14.995947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.195:42495)
0812 03:53:14.996158 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:14.996170 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:14.996240 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:14.996402 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:14.996403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:14.996581 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:14.996637 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:14.996639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.997363 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.997377 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:14.997924 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:14.997926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.998456 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.998457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:14.998527 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:14.999190 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:14.999196 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:15.001351 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
W20260812 03:53:15.001551 28131 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:14.996026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:42495 (local address 127.17.54.254:36705)
0812 03:53:14.996114 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:14.996116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:14.996223 (+ 107us) server_negotiation.cc:407] Connection header received
0812 03:53:14.996313 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:14.996314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:14.996328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:14.996372 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:14.996720 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.997247 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.998027 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:14.998219 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:14.998430 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:15.001466 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.195:42495: BlockingRecv error: recv got EOF from 127.17.54.195:42495 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 03:53:15 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:15 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.193@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:15.760203 28134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:15.755029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.193:57841)
0812 03:53:15.755170 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:15.755182 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:15.755282 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:15.755518 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:15.755519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:15.755693 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:15.755749 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:15.755751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:15.756513 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:15.756514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:15.757067 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:15.757069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:15.757454 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:15.757455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:15.757509 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:15.758018 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:15.758024 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:15.760092 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
W20260812 03:53:15.760252 28135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:15.755141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:57841 (local address 127.17.54.254:36705)
0812 03:53:15.755281 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:15.755283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:15.755292 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:53:15.755354 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:15.755355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:15.755370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:15.755472 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:15.755838 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:15.756357 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:15.757159 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:15.757358 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:15.757448 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:15.760182 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.193:57841: BlockingRecv error: recv got EOF from 127.17.54.193:57841 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
Aug 12 03:53:15 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:53:15 dist-test-slave-tm1g krb5kdc[26923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.54.194@KRBTEST.COM for oryx/127.17.54.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:53:15.879936 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:15.875063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36705 (local address 127.17.54.194:60657)
0812 03:53:15.875267 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:15.875280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:15.875369 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:15.875512 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:15.875513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:15.875700 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:15.875752 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:15.875754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:15.876456 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:15.876457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:15.877017 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:15.877019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:15.877420 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:15.877421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:15.877506 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:15.877968 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:15.877973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:15.879835 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.54.254:36705: Server oryx/127.17.54.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":107,"threads_started":1}
W20260812 03:53:15.879999 28135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:15.875167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:60657 (local address 127.17.54.254:36705)
0812 03:53:15.875247 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:15.875250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:15.875351 (+ 101us) server_negotiation.cc:407] Connection header received
0812 03:53:15.875429 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:15.875430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:15.875445 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:15.875506 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:15.875839 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:15.876358 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:15.877115 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:15.877323 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:15.877405 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:15.879935 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.54.194:60657: BlockingRecv error: recv got EOF from 127.17.54.194:60657 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:53:15.916491 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 27586
I20260812 03:53:15.924638 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 27761
I20260812 03:53:15.931670 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 27917
I20260812 03:53:15.938038 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 27505
2026-08-12T03:53:15Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17668 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T03:53:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:53:15Z Disabled control of system clock
I20260812 03:53:15.970443 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39587
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:41157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:39587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:53:16.059094 28141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:16.059319 28141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:16.059340 28141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:16.060611 28141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:53:16.060658 28141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:16.060671 28141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:16.060684 28141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:53:16.060698 28141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:53:16.062117 28141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:39587
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:16.062438 28141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:16.062778 28141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:53:16.068784 28141 server_base.cc:1034] running on GCE node
W20260812 03:53:16.068754 28147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.068743 28148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.068765 28150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:16.069413 28141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:16.069715 28141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:16.070878 28141 hybrid_clock.cc:648] HybridClock initialized: now 1786506796070854 us; error 45 us; skew 500 ppm
I20260812 03:53:16.072362 28141 webserver.cc:460] Webserver started at http://127.17.54.254:35023/ using document root <none> and password file <none>
I20260812 03:53:16.072609 28141 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:16.072659 28141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:16.072762 28141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:16.074002 28141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "cecd052ac7e1438596be52b9a0194199"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "67b3275522bf139817ce4741b9eb2592"
server_key_iv: "34673f5960698527c064b15b61bebf98"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.074447 28141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "cecd052ac7e1438596be52b9a0194199"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "67b3275522bf139817ce4741b9eb2592"
server_key_iv: "34673f5960698527c064b15b61bebf98"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.075995 28141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:16.076802 28156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.077055 28141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:16.077142 28141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "cecd052ac7e1438596be52b9a0194199"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "67b3275522bf139817ce4741b9eb2592"
server_key_iv: "34673f5960698527c064b15b61bebf98"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.077204 28141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:16.089143 28141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:16.089439 28141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:16.089556 28141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:16.093761 28141 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:39587
I20260812 03:53:16.093798 28219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:39587 every 8 connection(s)
I20260812 03:53:16.094183 28141 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:53:16.094995 28221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:53:16.095319 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 28141
I20260812 03:53:16.095563 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:53:16.095783 17627 external_mini_cluster.cc:1428] Setting key 4d990d7f089539b23de46d6b93c10fb8
I20260812 03:53:16.099141 28221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199: Bootstrap starting.
I20260812 03:53:16.099819 28221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199: Neither blocks nor log segments found. Creating new log.
I20260812 03:53:16.100171 28221 log.cc:824] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199: Log is configured to *not* fsync() on all Append() calls
I20260812 03:53:16.101043 28221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199: No bootstrap required, opened a new log
I20260812 03:53:16.102391 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.096393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.17.54.254:39587)
0812 03:53:16.096631 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:16.096641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:16.096677 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:53:16.097182 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:16.097192 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:16.097280 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:16.097451 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:16.097697 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.099086 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.100169 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.100768 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.100866 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:16.101602 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:16.101606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:16.101606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:53:16.101783 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:16.101817 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:53:16.101827 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:16.101886 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:16.102002 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:53:16.102065 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260812 03:53:16.103542 28221 raft_consensus.cc:348] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecd052ac7e1438596be52b9a0194199" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39587 } }
I20260812 03:53:16.103727 28221 raft_consensus.cc:374] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:53:16.103782 28221 raft_consensus.cc:729] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cecd052ac7e1438596be52b9a0194199, State: Initialized, Role: FOLLOWER
I20260812 03:53:16.103938 28221 consensus_queue.cc:260] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecd052ac7e1438596be52b9a0194199" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39587 } }
I20260812 03:53:16.104035 28221 raft_consensus.cc:388] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:53:16.104084 28221 raft_consensus.cc:482] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:53:16.104138 28221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:53:16.104770 28221 raft_consensus.cc:504] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecd052ac7e1438596be52b9a0194199" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39587 } }
I20260812 03:53:16.104918 28221 leader_election.cc:302] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cecd052ac7e1438596be52b9a0194199; no voters:
I20260812 03:53:16.105125 28221 leader_election.cc:288] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:53:16.105189 28227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:53:16.105499 28221 sys_catalog.cc:564] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:53:16.105657 28227 raft_consensus.cc:686] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [term 1 LEADER]: Becoming Leader. State: Replica: cecd052ac7e1438596be52b9a0194199, State: Running, Role: LEADER
I20260812 03:53:16.105808 28227 consensus_queue.cc:237] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecd052ac7e1438596be52b9a0194199" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39587 } }
I20260812 03:53:16.106196 28227 sys_catalog.cc:455] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cecd052ac7e1438596be52b9a0194199. Latest consensus state: current_term: 1 leader_uuid: "cecd052ac7e1438596be52b9a0194199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecd052ac7e1438596be52b9a0194199" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39587 } } }
I20260812 03:53:16.106547 28228 sys_catalog.cc:455] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cecd052ac7e1438596be52b9a0194199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecd052ac7e1438596be52b9a0194199" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39587 } } }
I20260812 03:53:16.106628 28228 sys_catalog.cc:458] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [sys.catalog]: This master's current role is: LEADER
I20260812 03:53:16.106274 28227 sys_catalog.cc:458] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199 [sys.catalog]: This master's current role is: LEADER
I20260812 03:53:16.107120 28236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:53:16.107703 28236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:53:16.109269 28236 catalog_manager.cc:1347] Generated new cluster ID: be68ba17921441feb6351181079f9234
I20260812 03:53:16.109334 28236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:53:16.141605 28236 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:53:16.142200 28236 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:53:16.147559 28236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cecd052ac7e1438596be52b9a0194199: Generated new TSK 0
I20260812 03:53:16.147773 28236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:53:16.160336 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39587
--builtin_ntp_servers=127.17.54.212:41157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 03:53:16.253438 28249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:16.253707 28249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:16.253748 28249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:16.254967 28249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:16.255051 28249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:16.255096 28249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:53:16.256520 28249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:16.256809 28249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:16.257135 28249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:16.263273 28254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.263537 28257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.263546 28255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:16.263513 28249 server_base.cc:1034] running on GCE node
I20260812 03:53:16.263880 28249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:16.264199 28249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:16.265408 28249 hybrid_clock.cc:648] HybridClock initialized: now 1786506796265407 us; error 58 us; skew 500 ppm
I20260812 03:53:16.266834 28249 webserver.cc:460] Webserver started at http://127.17.54.193:44957/ using document root <none> and password file <none>
I20260812 03:53:16.267107 28249 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:16.267155 28249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:16.267258 28249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:16.268428 28249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "447bac96248e47af84431a64b56d27e3"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "02232fd9f90828c07ccd1de35c1874a4"
server_key_iv: "77e45f1b5a6410eba3be2b65e7e10d26"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.268755 28249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "447bac96248e47af84431a64b56d27e3"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "02232fd9f90828c07ccd1de35c1874a4"
server_key_iv: "77e45f1b5a6410eba3be2b65e7e10d26"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.270121 28249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:53:16.271113 28265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.271375 28249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:16.271458 28249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "447bac96248e47af84431a64b56d27e3"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "02232fd9f90828c07ccd1de35c1874a4"
server_key_iv: "77e45f1b5a6410eba3be2b65e7e10d26"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.271517 28249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:16.293345 28249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:16.293661 28249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:16.293777 28249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:16.294008 28249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:16.294461 28249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:16.294505 28249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.294530 28249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:16.294574 28249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.301641 28249 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:42881
I20260812 03:53:16.301684 28404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:42881 every 8 connection(s)
I20260812 03:53:16.302131 28249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:53:16.307793 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 28249
I20260812 03:53:16.307906 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:53:16.308139 17627 external_mini_cluster.cc:1428] Setting key 280905f3d32202ea56e737c976325e8e
I20260812 03:53:16.308478 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.303061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:57865 (local address 127.17.54.254:39587)
0812 03:53:16.303128 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:16.303130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:16.303417 (+ 287us) server_negotiation.cc:407] Connection header received
0812 03:53:16.303929 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:16.303931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:16.303948 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:16.304018 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:16.304771 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.305317 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.306674 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.306924 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.306989 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:16.308098 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:16.308100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:16.308100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:53:16.308105 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:16.308123 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:53:16.308127 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:16.308182 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:16.308323 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 03:53:16.308356 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:53:16.308707 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.303027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:39587 (local address 127.17.54.193:57865)
0812 03:53:16.303245 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:16.303288 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:16.303816 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:16.304101 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:16.304105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:16.304114 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:16.304647 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:16.304652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.305445 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.305447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:16.306471 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:16.306475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.307026 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.307027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:16.307158 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:16.307829 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:53:16.307830 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:16.307874 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:53:16.307910 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:16.308213 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:16.308222 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:16.308225 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:53:16.308320 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:53:16.308447 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260812 03:53:16.309324 28405 heartbeater.cc:347] Connected to a master server at 127.17.54.254:39587
I20260812 03:53:16.309358 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39587
--builtin_ntp_servers=127.17.54.212:41157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:53:16.309422 28405 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:16.309752 28405 heartbeater.cc:511] Master 127.17.54.254:39587 requested a full tablet report, sending...
I20260812 03:53:16.310446 28174 ts_manager.cc:194] Registered new tserver with Master: 447bac96248e47af84431a64b56d27e3 (127.17.54.193:42881)
I20260812 03:53:16.311085 28174 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.193:57865
W20260812 03:53:16.399461 28409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:16.399689 28409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:16.399712 28409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:16.400861 28409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:16.400899 28409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:16.400925 28409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:53:16.402236 28409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:16.402522 28409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:16.402799 28409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:16.408494 28414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.408519 28415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.408564 28417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:16.408785 28409 server_base.cc:1034] running on GCE node
I20260812 03:53:16.409031 28409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:16.409293 28409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:16.410468 28409 hybrid_clock.cc:648] HybridClock initialized: now 1786506796410448 us; error 51 us; skew 500 ppm
I20260812 03:53:16.411907 28409 webserver.cc:460] Webserver started at http://127.17.54.194:42235/ using document root <none> and password file <none>
I20260812 03:53:16.412206 28409 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:16.412286 28409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:16.412451 28409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:16.413657 28409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "166becf0f3374bcbb8a4c2648f137ffc"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "87438b430107400d3479ff5747b9c562"
server_key_iv: "6d599f2c330f20f13e7b7a3a2d2f52d0"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.414062 28409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "166becf0f3374bcbb8a4c2648f137ffc"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "87438b430107400d3479ff5747b9c562"
server_key_iv: "6d599f2c330f20f13e7b7a3a2d2f52d0"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.415555 28409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:53:16.416471 28423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.416668 28409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:16.416777 28409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "166becf0f3374bcbb8a4c2648f137ffc"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "87438b430107400d3479ff5747b9c562"
server_key_iv: "6d599f2c330f20f13e7b7a3a2d2f52d0"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.416868 28409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:16.429034 28409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:16.429389 28409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:16.429555 28409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:16.429818 28409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:16.430192 28409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:16.430253 28409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.430446 28409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:16.430516 28409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.437726 28409 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:44875
I20260812 03:53:16.437768 28558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:44875 every 8 connection(s)
I20260812 03:53:16.438156 28409 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:53:16.444212 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.439088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:37717 (local address 127.17.54.254:39587)
0812 03:53:16.439178 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:16.439181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:16.439437 (+ 256us) server_negotiation.cc:407] Connection header received
0812 03:53:16.439936 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:16.439937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:16.439947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:16.440013 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:16.440904 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.441588 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.442751 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.442947 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.443006 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:16.443858 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:16.443860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:16.443860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:53:16.443865 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:16.443881 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:53:16.443885 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:16.443938 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:16.444120 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 03:53:16.444145 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:53:16.444425 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.439098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:39587 (local address 127.17.54.194:37717)
0812 03:53:16.439301 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:16.439343 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:16.439847 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:16.440066 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:16.440069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:16.440076 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:16.440772 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:16.440779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.441713 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.441714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:16.442668 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:16.442671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.443030 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.443031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:16.443139 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:16.443715 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:53:16.443716 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:16.443745 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:53:16.443769 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:16.443987 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:16.443995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:16.443997 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:53:16.444087 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:53:16.444179 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260812 03:53:16.445103 28560 heartbeater.cc:347] Connected to a master server at 127.17.54.254:39587
I20260812 03:53:16.445205 28560 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:16.445387 28560 heartbeater.cc:511] Master 127.17.54.254:39587 requested a full tablet report, sending...
I20260812 03:53:16.445844 28174 ts_manager.cc:194] Registered new tserver with Master: 166becf0f3374bcbb8a4c2648f137ffc (127.17.54.194:44875)
I20260812 03:53:16.445883 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 28409
I20260812 03:53:16.445991 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:53:16.446187 17627 external_mini_cluster.cc:1428] Setting key ad69a1692b2d6a271e53d57d6d93ef48
I20260812 03:53:16.446346 28174 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.194:37717
I20260812 03:53:16.447237 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39587
--builtin_ntp_servers=127.17.54.212:41157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 03:53:16.537063 28563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:16.537308 28563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:16.537334 28563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:16.538733 28563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:16.538780 28563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:16.538808 28563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:53:16.540277 28563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:16.540510 28563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:16.540802 28563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:16.547297 28570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.547308 28569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:16.547328 28563 server_base.cc:1034] running on GCE node
W20260812 03:53:16.547305 28572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:16.547760 28563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:16.548054 28563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:16.549238 28563 hybrid_clock.cc:648] HybridClock initialized: now 1786506796549242 us; error 48 us; skew 500 ppm
I20260812 03:53:16.550760 28563 webserver.cc:460] Webserver started at http://127.17.54.195:43245/ using document root <none> and password file <none>
I20260812 03:53:16.551045 28563 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:16.551124 28563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:16.551281 28563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:16.552383 28563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "30e6a670c8964202b3dd9c67da5b4b71"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "091e061e383041ee9aedb71f6c16a4ae"
server_key_iv: "aaec88db4be819e1297be7649ebe9716"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.552770 28563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "30e6a670c8964202b3dd9c67da5b4b71"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "091e061e383041ee9aedb71f6c16a4ae"
server_key_iv: "aaec88db4be819e1297be7649ebe9716"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.554450 28563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:53:16.555356 28578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.555588 28563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:53:16.555698 28563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "30e6a670c8964202b3dd9c67da5b4b71"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "091e061e383041ee9aedb71f6c16a4ae"
server_key_iv: "aaec88db4be819e1297be7649ebe9716"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.555786 28563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:16.569383 28563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:16.569764 28563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:16.569952 28563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:16.570201 28563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:16.570626 28563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:16.570686 28563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.570734 28563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:16.570780 28563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.577003 28563 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:34367
I20260812 03:53:16.577078 28705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:34367 every 8 connection(s)
I20260812 03:53:16.577445 28563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:53:16.583606 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 28563
I20260812 03:53:16.583665 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.578290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:40759 (local address 127.17.54.254:39587)
0812 03:53:16.578378 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:16.578381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:16.578637 (+ 256us) server_negotiation.cc:407] Connection header received
0812 03:53:16.579243 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:16.579244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:16.579259 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:16.579311 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:16.580078 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.580863 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.582002 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.582217 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.582299 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:16.583307 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:16.583309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:16.583309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:53:16.583315 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:16.583332 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:53:16.583337 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:16.583410 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:16.583518 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 03:53:16.583579 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:53:16.583719 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:53:16.583886 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.578269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:39587 (local address 127.17.54.195:40759)
0812 03:53:16.578508 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:16.578544 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:16.579114 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:16.579381 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:16.579385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:16.579401 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:16.579898 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:16.579902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.580998 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.580999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:16.581876 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:16.581879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:16.582356 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:16.582357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:16.582498 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:16.583096 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:53:16.583097 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:16.583128 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:53:16.583152 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:16.583412 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:16.583419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:16.583423 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:53:16.583519 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:53:16.583638 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":122,"threads_started":1}
I20260812 03:53:16.583952 17627 external_mini_cluster.cc:1428] Setting key 23342c34121a6bc4b0c79d35463c8e84
I20260812 03:53:16.584501 28706 heartbeater.cc:347] Connected to a master server at 127.17.54.254:39587
I20260812 03:53:16.584630 28706 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:16.584867 28706 heartbeater.cc:511] Master 127.17.54.254:39587 requested a full tablet report, sending...
I20260812 03:53:16.585402 28173 ts_manager.cc:194] Registered new tserver with Master: 30e6a670c8964202b3dd9c67da5b4b71 (127.17.54.195:34367)
I20260812 03:53:16.585857 28173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.195:40759
I20260812 03:53:16.596199 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:53:16.598500 28225 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:16.597725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49510 (local address 127.17.54.254:39587)
0812 03:53:16.597803 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:16.597809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:16.597946 (+ 137us) server_negotiation.cc:407] Connection header received
0812 03:53:16.597993 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:16.597994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:16.598003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:16.598058 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:16.598364 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49510: BlockingRecv error: recv got EOF from 127.0.0.1:49510 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:53:16.598502 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 28249
I20260812 03:53:16.605481 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 28409
I20260812 03:53:16.611783 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 28563
I20260812 03:53:16.620079 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 28141
2026-08-12T03:53:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (680 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T03:53:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:53:16Z Disabled control of system clock
I20260812 03:53:16.651464 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42913
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:42913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:53:16.735344 28718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:16.735554 28718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:16.735576 28718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:16.736760 28718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:53:16.736807 28718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:16.736820 28718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:16.736832 28718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:53:16.736845 28718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:53:16.738221 28718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:42913
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:16.738520 28718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:16.738788 28718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:16.744189 28725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.744287 28726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.744258 28728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:16.744782 28718 server_base.cc:1034] running on GCE node
I20260812 03:53:16.745018 28718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:16.745283 28718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:16.746477 28718 hybrid_clock.cc:648] HybridClock initialized: now 1786506796746445 us; error 38 us; skew 500 ppm
I20260812 03:53:16.747852 28718 webserver.cc:460] Webserver started at http://127.17.54.254:45299/ using document root <none> and password file <none>
I20260812 03:53:16.748116 28718 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:16.748167 28718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:16.748304 28718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:16.749476 28718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "83ccb5bd0e7c4ffe18873b605880fbc8"
server_key_iv: "d8cd418bc2fc4fe8f3e6b05508edc47c"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.749867 28718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "83ccb5bd0e7c4ffe18873b605880fbc8"
server_key_iv: "d8cd418bc2fc4fe8f3e6b05508edc47c"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.751437 28718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:53:16.752383 28734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.752607 28718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:53:16.752727 28718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "83ccb5bd0e7c4ffe18873b605880fbc8"
server_key_iv: "d8cd418bc2fc4fe8f3e6b05508edc47c"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.752813 28718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:16.756963 28718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:16.757273 28718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:16.757442 28718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:16.761924 28718 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:42913
I20260812 03:53:16.761969 28796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:42913 every 8 connection(s)
I20260812 03:53:16.762413 28718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:53:16.763212 28798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:53:16.766276 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 28718
I20260812 03:53:16.766422 28798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074: Bootstrap starting.
I20260812 03:53:16.766471 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:53:16.766662 17627 external_mini_cluster.cc:1428] Setting key a9e69f97245665d432ad114a72aad1e2
I20260812 03:53:16.767261 28798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074: Neither blocks nor log segments found. Creating new log.
I20260812 03:53:16.767690 28798 log.cc:824] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074: Log is configured to *not* fsync() on all Append() calls
I20260812 03:53:16.768658 28798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074: No bootstrap required, opened a new log
I20260812 03:53:16.769924 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.767279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43906 (local address 127.17.54.254:42913)
0812 03:53:16.767581 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:16.767592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:16.767631 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:53:16.768380 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:16.768389 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:16.768470 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:16.768600 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:16.769379 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:16.769382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:16.769383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:53:16.769387 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:16.769438 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:53:16.769449 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:16.769529 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:16.769613 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:53:16.769701 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":151,"threads_started":1}
I20260812 03:53:16.770709 28798 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42913 } }
I20260812 03:53:16.770877 28798 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:53:16.770902 28798 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1fcbd1f0a0c4e42a8f6a24d01766074, State: Initialized, Role: FOLLOWER
I20260812 03:53:16.771045 28798 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42913 } }
I20260812 03:53:16.771118 28798 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:53:16.771140 28798 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:53:16.771168 28798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:53:16.772008 28798 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42913 } }
I20260812 03:53:16.772097 28798 leader_election.cc:302] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1fcbd1f0a0c4e42a8f6a24d01766074; no voters:
I20260812 03:53:16.772228 28798 leader_election.cc:288] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:53:16.772312 28804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:53:16.772505 28798 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:53:16.772511 28804 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [term 1 LEADER]: Becoming Leader. State: Replica: c1fcbd1f0a0c4e42a8f6a24d01766074, State: Running, Role: LEADER
I20260812 03:53:16.772744 28804 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42913 } }
I20260812 03:53:16.773140 28805 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42913 } } }
I20260812 03:53:16.773229 28805 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [sys.catalog]: This master's current role is: LEADER
I20260812 03:53:16.773643 28806 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1fcbd1f0a0c4e42a8f6a24d01766074. Latest consensus state: current_term: 1 leader_uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fcbd1f0a0c4e42a8f6a24d01766074" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42913 } } }
I20260812 03:53:16.773757 28806 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074 [sys.catalog]: This master's current role is: LEADER
I20260812 03:53:16.773844 28813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:53:16.774390 28813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:53:16.776098 28813 catalog_manager.cc:1347] Generated new cluster ID: e5ff761f629f468f9162e23c9095faff
I20260812 03:53:16.776150 28813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:53:16.789682 28813 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:53:16.789896 28813 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:53:16.794284 28813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1fcbd1f0a0c4e42a8f6a24d01766074: Generated new TSK 0
I20260812 03:53:16.794524 28813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:53:16.804425 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42913
--builtin_ntp_servers=127.17.54.212:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:53:16.891588 28824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:16.891815 28824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:16.891840 28824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:16.893431 28824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:16.893497 28824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:16.893532 28824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:53:16.895284 28824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:16.895593 28824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:16.895963 28824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:16.902240 28832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.902240 28834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:16.902278 28830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:16.903090 28824 server_base.cc:1034] running on GCE node
I20260812 03:53:16.903357 28824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:16.903621 28824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:16.904814 28824 hybrid_clock.cc:648] HybridClock initialized: now 1786506796904787 us; error 41 us; skew 500 ppm
I20260812 03:53:16.906293 28824 webserver.cc:460] Webserver started at http://127.17.54.193:42867/ using document root <none> and password file <none>
I20260812 03:53:16.906715 28824 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:16.906800 28824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:16.906982 28824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:16.908284 28824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "a5329e698fec493fbc9bb32e6f5ac843"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "d5cbfbc5bfe733f397fa0f7fed4e7fdc"
server_key_iv: "19c1bc85b847c217a70b6b222d472c95"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.908720 28824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "a5329e698fec493fbc9bb32e6f5ac843"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "d5cbfbc5bfe733f397fa0f7fed4e7fdc"
server_key_iv: "19c1bc85b847c217a70b6b222d472c95"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.910290 28824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 03:53:16.911311 28840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.911527 28824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:16.911643 28824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "a5329e698fec493fbc9bb32e6f5ac843"
format_stamp: "Formatted at 2026-08-12 03:53:16 on dist-test-slave-tm1g"
server_key: "d5cbfbc5bfe733f397fa0f7fed4e7fdc"
server_key_iv: "19c1bc85b847c217a70b6b222d472c95"
server_key_version: "encryptionkey@0"
I20260812 03:53:16.911737 28824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:16.915334 28824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:16.915637 28824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:16.915800 28824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:16.916035 28824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:16.916371 28824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:16.916429 28824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.916469 28824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:16.916497 28824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:16.922798 28824 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:37519
I20260812 03:53:16.922921 28977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:37519 every 8 connection(s)
I20260812 03:53:16.923245 28824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:53:16.926750 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.924081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:46729 (local address 127.17.54.254:42913)
0812 03:53:16.924199 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:16.924201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:16.924445 (+ 244us) server_negotiation.cc:407] Connection header received
0812 03:53:16.924967 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:16.924968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:16.924975 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:16.925040 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:16.926293 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:16.926295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:16.926295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:53:16.926374 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:16.926394 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:53:16.926399 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:16.926475 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:16.926678 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 03:53:16.926691 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 03:53:16.926957 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:16.924074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42913 (local address 127.17.54.193:46729)
0812 03:53:16.924303 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:16.924345 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:16.924842 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:16.925092 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:16.925096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:16.925105 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:16.926115 (+ 1010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:53:16.926115 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:16.926145 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:53:16.926175 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:16.926576 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:16.926584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:16.926587 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:53:16.926669 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:53:16.926725 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":130,"threads_started":1}
I20260812 03:53:16.927663 28978 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42913
I20260812 03:53:16.927764 28978 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:16.927907 28978 heartbeater.cc:511] Master 127.17.54.254:42913 requested a full tablet report, sending...
I20260812 03:53:16.928521 28755 ts_manager.cc:194] Registered new tserver with Master: a5329e698fec493fbc9bb32e6f5ac843 (127.17.54.193:37519)
I20260812 03:53:16.930662 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 28824
I20260812 03:53:16.930743 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:53:16.930950 17627 external_mini_cluster.cc:1428] Setting key ffe1d1ef95cd19d9bdd02555c76455f6
I20260812 03:53:16.932045 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42913
--builtin_ntp_servers=127.17.54.212:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:53:17.016741 28982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:17.017045 28982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:17.017091 28982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:17.018414 28982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:17.018484 28982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:17.018530 28982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:53:17.020008 28982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:17.020313 28982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:17.020650 28982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:17.026399 28991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:17.026433 28987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:17.026432 28988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:17.026800 28982 server_base.cc:1034] running on GCE node
I20260812 03:53:17.027068 28982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:17.027364 28982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:17.028573 28982 hybrid_clock.cc:648] HybridClock initialized: now 1786506797028521 us; error 106 us; skew 500 ppm
I20260812 03:53:17.030156 28982 webserver.cc:460] Webserver started at http://127.17.54.194:36115/ using document root <none> and password file <none>
I20260812 03:53:17.030511 28982 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:17.030568 28982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:17.030752 28982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:17.031934 28982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "04bdda4668f44529aedb60bed8565524"
format_stamp: "Formatted at 2026-08-12 03:53:17 on dist-test-slave-tm1g"
server_key: "14dd1d0df367eb10b563bc7b8ae6a382"
server_key_iv: "45f203af71dafa422ce02dfa5c3ff183"
server_key_version: "encryptionkey@0"
I20260812 03:53:17.032338 28982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "04bdda4668f44529aedb60bed8565524"
format_stamp: "Formatted at 2026-08-12 03:53:17 on dist-test-slave-tm1g"
server_key: "14dd1d0df367eb10b563bc7b8ae6a382"
server_key_iv: "45f203af71dafa422ce02dfa5c3ff183"
server_key_version: "encryptionkey@0"
I20260812 03:53:17.034052 28982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:53:17.035221 28999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:17.035575 28982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:53:17.035718 28982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "04bdda4668f44529aedb60bed8565524"
format_stamp: "Formatted at 2026-08-12 03:53:17 on dist-test-slave-tm1g"
server_key: "14dd1d0df367eb10b563bc7b8ae6a382"
server_key_iv: "45f203af71dafa422ce02dfa5c3ff183"
server_key_version: "encryptionkey@0"
I20260812 03:53:17.035836 28982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:17.039613 28982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:17.039922 28982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:17.040076 28982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:17.040444 28982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:17.040879 28982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:17.040952 28982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:17.041000 28982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:17.041023 28982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:17.047996 28982 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:37803
I20260812 03:53:17.048055 29127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:37803 every 8 connection(s)
I20260812 03:53:17.048473 28982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:53:17.051950 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:17.049396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:44535 (local address 127.17.54.254:42913)
0812 03:53:17.049461 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:17.049463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:17.049667 (+ 204us) server_negotiation.cc:407] Connection header received
0812 03:53:17.050238 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:17.050239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:17.050247 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:17.050374 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:17.051649 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:17.051651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:17.051651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:53:17.051656 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:17.051678 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:53:17.051683 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:17.051752 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:17.051853 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:53:17.051870 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:53:17.052207 29130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:17.049358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42913 (local address 127.17.54.194:44535)
0812 03:53:17.049536 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:17.049572 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:17.050105 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:17.050417 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:17.050422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:17.050432 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:17.051456 (+ 1024us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:53:17.051456 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:17.051487 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:53:17.051516 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:17.051751 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:17.051758 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:17.051762 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:53:17.051891 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 03:53:17.051959 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260812 03:53:17.052728 29128 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42913
I20260812 03:53:17.052811 29128 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:17.053009 29128 heartbeater.cc:511] Master 127.17.54.254:42913 requested a full tablet report, sending...
I20260812 03:53:17.053380 28755 ts_manager.cc:194] Registered new tserver with Master: 04bdda4668f44529aedb60bed8565524 (127.17.54.194:37803)
I20260812 03:53:17.058135 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 28982
I20260812 03:53:17.058225 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:53:17.058513 17627 external_mini_cluster.cc:1428] Setting key 3ef73727d94dc13a9f499651a0cc89a8
I20260812 03:53:17.059787 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42913
--builtin_ntp_servers=127.17.54.212:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:53:17.147831 29132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:17.148101 29132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:17.148144 29132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:17.149396 29132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:17.149478 29132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:17.149528 29132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:53:17.151086 29132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:17.151386 29132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:17.151716 29132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:17.157742 29137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:17.157742 29138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:17.157763 29141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:17.158733 29132 server_base.cc:1034] running on GCE node
I20260812 03:53:17.159309 29132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:17.159622 29132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:17.160940 29132 hybrid_clock.cc:648] HybridClock initialized: now 1786506797160923 us; error 36 us; skew 500 ppm
I20260812 03:53:17.162632 29132 webserver.cc:460] Webserver started at http://127.17.54.195:42815/ using document root <none> and password file <none>
I20260812 03:53:17.162920 29132 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:17.163002 29132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:17.163173 29132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:17.164362 29132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "79de13bab6214e4b87eae52fcf99ecde"
format_stamp: "Formatted at 2026-08-12 03:53:17 on dist-test-slave-tm1g"
server_key: "93d03126cea31f334c500684aa71c9ad"
server_key_iv: "9e46cee16495ab146462f8a76baf09b9"
server_key_version: "encryptionkey@0"
I20260812 03:53:17.164760 29132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "79de13bab6214e4b87eae52fcf99ecde"
format_stamp: "Formatted at 2026-08-12 03:53:17 on dist-test-slave-tm1g"
server_key: "93d03126cea31f334c500684aa71c9ad"
server_key_iv: "9e46cee16495ab146462f8a76baf09b9"
server_key_version: "encryptionkey@0"
I20260812 03:53:17.166602 29132 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 03:53:17.167553 29148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:17.167799 29132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:17.167932 29132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "79de13bab6214e4b87eae52fcf99ecde"
format_stamp: "Formatted at 2026-08-12 03:53:17 on dist-test-slave-tm1g"
server_key: "93d03126cea31f334c500684aa71c9ad"
server_key_iv: "9e46cee16495ab146462f8a76baf09b9"
server_key_version: "encryptionkey@0"
I20260812 03:53:17.168022 29132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:17.171886 29132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:17.172255 29132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:17.172420 29132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:17.172716 29132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:17.173054 29132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:17.173118 29132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:17.173159 29132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:17.173187 29132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:17.180233 29132 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:33113
I20260812 03:53:17.180281 29280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:33113 every 8 connection(s)
I20260812 03:53:17.180648 29132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:53:17.184176 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:17.181469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:60425 (local address 127.17.54.254:42913)
0812 03:53:17.181543 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:17.181545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:17.181846 (+ 301us) server_negotiation.cc:407] Connection header received
0812 03:53:17.182547 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:17.182548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:17.182556 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:17.182616 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:17.183847 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:17.183848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:17.183848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:53:17.183852 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:17.183868 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:53:17.183873 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:17.183932 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:17.184022 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:53:17.184036 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:53:17.184269 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:17.181421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42913 (local address 127.17.54.195:60425)
0812 03:53:17.181644 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:17.181687 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:17.182393 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:17.182701 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:17.182705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:17.182713 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:17.183666 (+ 953us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:53:17.183666 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:17.183697 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:53:17.183726 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:17.183945 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:17.183950 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:17.183952 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:53:17.184013 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:53:17.184066 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260812 03:53:17.184913 29282 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42913
I20260812 03:53:17.185004 29282 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:17.185207 29282 heartbeater.cc:511] Master 127.17.54.254:42913 requested a full tablet report, sending...
I20260812 03:53:17.185771 28755 ts_manager.cc:194] Registered new tserver with Master: 79de13bab6214e4b87eae52fcf99ecde (127.17.54.195:33113)
I20260812 03:53:17.186266 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 29132
I20260812 03:53:17.186466 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:53:17.186661 17627 external_mini_cluster.cc:1428] Setting key b9fa1b0ce4893519667a2cae805be387
I20260812 03:53:17.187844 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:53:17.190197 28802 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:53:17.189730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.17.54.254:42913)
0812 03:53:17.189836 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:17.189838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:17.189850 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:53:17.189875 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:17.189876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:17.189880 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:17.189964 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:17.190127 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43916: BlockingRecv error: recv got EOF from 127.0.0.1:43916 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:53:17.190412 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 28824
I20260812 03:53:17.203624 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 28982
I20260812 03:53:17.211759 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 29132
I20260812 03:53:17.221937 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 28718
2026-08-12T03:53:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (602 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:53:17 dist-test-slave-tm1g krb5kdc[29293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:53:17 dist-test-slave-tm1g krb5kdc[29293](info): set up 2 sockets
Aug 12 03:53:17 dist-test-slave-tm1g krb5kdc[29293](info): commencing operation
krb5kdc: starting...
W20260812 03:53:18.348912 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:53:18 dist-test-slave-tm1g krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:53:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:53:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:53:18.486409 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:36155
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:36155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:53:18.571669 29313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:18.571943 29313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:18.571981 29313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:18.573161 29313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:53:18.573248 29313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:18.573275 29313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:18.573305 29313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:53:18.573328 29313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:53:18.574985 29313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:36155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:36155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:18.575284 29313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:18.575551 29313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:18.580988 29318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:18.581029 29319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:18.581095 29313 server_base.cc:1034] running on GCE node
W20260812 03:53:18.581032 29321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:18.581431 29313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:18.581768 29313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:18.582942 29313 hybrid_clock.cc:648] HybridClock initialized: now 1786506798582922 us; error 47 us; skew 500 ppm
Aug 12 03:53:18 dist-test-slave-tm1g krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506798, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:18.585899 29313 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:53:18.586513 29313 webserver.cc:460] Webserver started at http://127.17.54.254:41167/ using document root <none> and password file <none>
I20260812 03:53:18.586800 29313 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:18.586922 29313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:18.587108 29313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:18.588265 29313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "a193dc53ce3345858459e5faeacc3d22"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "f7d30dbf0bcecc9af68c472ecb05b61d"
server_key_iv: "4864db951225ae75274e22de0df4f762"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.588708 29313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "a193dc53ce3345858459e5faeacc3d22"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "f7d30dbf0bcecc9af68c472ecb05b61d"
server_key_iv: "4864db951225ae75274e22de0df4f762"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.590204 29313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:53:18.591089 29329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:18.591344 29313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:18.591498 29313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "a193dc53ce3345858459e5faeacc3d22"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "f7d30dbf0bcecc9af68c472ecb05b61d"
server_key_iv: "4864db951225ae75274e22de0df4f762"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.591590 29313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:18.606854 29313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:18.609519 29313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:18.609714 29313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:18.614259 29313 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:36155
I20260812 03:53:18.614346 29395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:36155 every 8 connection(s)
I20260812 03:53:18.614760 29313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:53:18.615911 29396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:53:18.618533 29396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22: Bootstrap starting.
I20260812 03:53:18.619124 29396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22: Neither blocks nor log segments found. Creating new log.
I20260812 03:53:18.619374 29396 log.cc:824] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22: Log is configured to *not* fsync() on all Append() calls
I20260812 03:53:18.620218 29396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22: No bootstrap required, opened a new log
I20260812 03:53:18.622422 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 29313
I20260812 03:53:18.622526 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:53:18.622395 29396 raft_consensus.cc:348] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a193dc53ce3345858459e5faeacc3d22" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36155 } }
I20260812 03:53:18.622604 29396 raft_consensus.cc:374] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:53:18.622653 29396 raft_consensus.cc:729] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a193dc53ce3345858459e5faeacc3d22, State: Initialized, Role: FOLLOWER
I20260812 03:53:18.622718 17627 external_mini_cluster.cc:1428] Setting key ddf9279521e4e6b0dca66d04e12f9c37
I20260812 03:53:18.622814 29396 consensus_queue.cc:260] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a193dc53ce3345858459e5faeacc3d22" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36155 } }
I20260812 03:53:18.622905 29396 raft_consensus.cc:388] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:53:18.622937 29396 raft_consensus.cc:482] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:53:18.622991 29396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:53:18.624253 29396 raft_consensus.cc:504] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a193dc53ce3345858459e5faeacc3d22" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36155 } }
I20260812 03:53:18.624389 29396 leader_election.cc:302] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a193dc53ce3345858459e5faeacc3d22; no voters:
I20260812 03:53:18.624573 29396 leader_election.cc:288] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:53:18.624614 29401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:53:18.624820 29401 raft_consensus.cc:686] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [term 1 LEADER]: Becoming Leader. State: Replica: a193dc53ce3345858459e5faeacc3d22, State: Running, Role: LEADER
I20260812 03:53:18.624856 29396 sys_catalog.cc:564] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:53:18.624927 29401 consensus_queue.cc:237] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a193dc53ce3345858459e5faeacc3d22" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36155 } }
I20260812 03:53:18.625406 29403 sys_catalog.cc:455] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a193dc53ce3345858459e5faeacc3d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a193dc53ce3345858459e5faeacc3d22" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36155 } } }
I20260812 03:53:18.625560 29403 sys_catalog.cc:458] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [sys.catalog]: This master's current role is: LEADER
I20260812 03:53:18.626237 29408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:53:18.626471 29402 sys_catalog.cc:455] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a193dc53ce3345858459e5faeacc3d22. Latest consensus state: current_term: 1 leader_uuid: "a193dc53ce3345858459e5faeacc3d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a193dc53ce3345858459e5faeacc3d22" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36155 } } }
I20260812 03:53:18.626703 29402 sys_catalog.cc:458] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22 [sys.catalog]: This master's current role is: LEADER
I20260812 03:53:18.627029 29408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:53:18.628621 29408 catalog_manager.cc:1347] Generated new cluster ID: 09e61dcab72249ef9be89b355af31f3f
I20260812 03:53:18.628683 29408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:53:18 dist-test-slave-tm1g krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:53:18.635252 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:18.623279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.17.54.254:36155)
0812 03:53:18.623480 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:18.623488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:18.623517 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:53:18.623655 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:18.623661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:18.623707 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:18.623828 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:18.624683 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:18.626277 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:18.627504 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:18.628135 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:18.628252 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:18.631292 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:18.631297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:18.631297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:53:18.631311 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:18.633404 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:18.633825 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:18.633829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:18.633829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:18.633968 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:18.634190 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:18.634191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:18.634191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:18.634586 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:18.634796 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:18.634872 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:53:18.635035 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260812 03:53:18.643527 29408 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:53:18.644280 29408 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:53:18.651633 29408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22: Generated new TSK 0
I20260812 03:53:18.651885 29408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:53:18.690717 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36155
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:53:18.778034 29425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:18.778249 29425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:18.778275 29425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:18.779547 29425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:18.779596 29425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:18.779625 29425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:53:18.781180 29425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:18.781476 29425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:18.781750 29425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:18.787561 29430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:18.787561 29431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:18.787856 29425 server_base.cc:1034] running on GCE node
W20260812 03:53:18.787564 29434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:18.788211 29425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:18.788552 29425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:18.789776 29425 hybrid_clock.cc:648] HybridClock initialized: now 1786506798789774 us; error 45 us; skew 500 ppm
Aug 12 03:53:18 dist-test-slave-tm1g krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506798, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:18.796285 29425 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:53:18.796802 29425 webserver.cc:460] Webserver started at http://127.17.54.193:41655/ using document root <none> and password file <none>
I20260812 03:53:18.797083 29425 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:18.797142 29425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:18.797281 29425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:18.798470 29425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "68f65cc0722846d68f7af52747b758f9"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "b37f49bedbcb4e842625203923b90652"
server_key_iv: "134fe83a3a41332d6d0683413f02a76b"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.798853 29425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "68f65cc0722846d68f7af52747b758f9"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "b37f49bedbcb4e842625203923b90652"
server_key_iv: "134fe83a3a41332d6d0683413f02a76b"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.800282 29425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:18.801132 29442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:18.801354 29425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:18.801441 29425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "68f65cc0722846d68f7af52747b758f9"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "b37f49bedbcb4e842625203923b90652"
server_key_iv: "134fe83a3a41332d6d0683413f02a76b"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.801530 29425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:18.813617 29425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:18.816180 29425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:18.816316 29425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:18.816534 29425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:18.816891 29425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:18.816972 29425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:18.817013 29425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:18.817063 29425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:18.823998 29425 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:42221
I20260812 03:53:18.824090 29583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:42221 every 8 connection(s)
I20260812 03:53:18.824425 29425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:53:18.827261 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 29425
I20260812 03:53:18.827343 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:53:18.827577 17627 external_mini_cluster.cc:1428] Setting key 99556394f1e164ae0c0f0a1309932c78
Aug 12 03:53:18 dist-test-slave-tm1g krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506798, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:53:18.837076 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:18.825362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:44527 (local address 127.17.54.254:36155)
0812 03:53:18.825442 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:18.825445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:18.825807 (+ 362us) server_negotiation.cc:407] Connection header received
0812 03:53:18.826271 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:18.826272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:18.826284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:18.826454 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:18.827608 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:18.828313 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:18.829971 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:18.830251 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:18.830477 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:18.833359 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:18.833361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:18.833362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:53:18.833372 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:18.835384 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:18.835874 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:18.835876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:18.835876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:18.835951 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:18.836244 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:18.836246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:18.836247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:18.836421 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:18.836494 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:18.836866 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 03:53:18.836962 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:53:18.837186 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:18.825339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36155 (local address 127.17.54.193:44527)
0812 03:53:18.825543 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:18.825657 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:18.826183 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:18.826509 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:18.826512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:18.827214 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:18.827502 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:18.827509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:18.828437 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:18.828439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:18.829790 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:18.829794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:18.830509 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:18.830510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:18.830646 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:18.831260 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:18.831269 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:18.833225 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:18.835495 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:18.835505 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:18.835507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:18.835779 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:18.836038 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:18.836039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:18.836040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:18.836164 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:18.836539 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:18.836546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:18.836691 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 03:53:18.836799 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:53:18.836949 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260812 03:53:18.837841 29584 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36155
I20260812 03:53:18.838086 29584 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
I20260812 03:53:18.838344 29584 heartbeater.cc:511] Master 127.17.54.254:36155 requested a full tablet report, sending...
I20260812 03:53:18.838900 29351 ts_manager.cc:194] Registered new tserver with Master: 68f65cc0722846d68f7af52747b758f9 (127.17.54.193:42221)
I20260812 03:53:18.839612 29351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:44527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:53:18.863498 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36155
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:53:18.966616 29591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:18.966845 29591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:18.966871 29591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:18.968034 29591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:18.968070 29591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:18.968096 29591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:53:18.969558 29591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:18.969755 29591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:18.969954 29591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:18.975535 29599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:18.975708 29591 server_base.cc:1034] running on GCE node
W20260812 03:53:18.975538 29596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:18.975535 29597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:18.976004 29591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:18.976291 29591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:18.977510 29591 hybrid_clock.cc:648] HybridClock initialized: now 1786506798977492 us; error 28 us; skew 500 ppm
Aug 12 03:53:18 dist-test-slave-tm1g krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506798, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:18.980281 29591 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:53:18.980794 29591 webserver.cc:460] Webserver started at http://127.17.54.194:34815/ using document root <none> and password file <none>
I20260812 03:53:18.981051 29591 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:18.981132 29591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:18.981302 29591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:18.982539 29591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "ce1c4deea82744e5adbd247719db25c4"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "bbec6cbdde98086bdbe8776c1df66365"
server_key_iv: "5a373f0ce50b5776216c480d66ab270a"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.982931 29591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "ce1c4deea82744e5adbd247719db25c4"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "bbec6cbdde98086bdbe8776c1df66365"
server_key_iv: "5a373f0ce50b5776216c480d66ab270a"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.984505 29591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:53:18.985428 29606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:18.985620 29591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:18.985747 29591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "ce1c4deea82744e5adbd247719db25c4"
format_stamp: "Formatted at 2026-08-12 03:53:18 on dist-test-slave-tm1g"
server_key: "bbec6cbdde98086bdbe8776c1df66365"
server_key_iv: "5a373f0ce50b5776216c480d66ab270a"
server_key_version: "encryptionkey@0"
I20260812 03:53:18.985837 29591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:19.012378 29591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:19.014806 29591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:19.014997 29591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:19.015282 29591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:19.015657 29591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:19.015722 29591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:19.015764 29591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:19.015793 29591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:19.022993 29591 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:39991
I20260812 03:53:19.023043 29739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:39991 every 8 connection(s)
I20260812 03:53:19.023422 29591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:53:19.030268 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 29591
I20260812 03:53:19.030421 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:53:19.030664 17627 external_mini_cluster.cc:1428] Setting key 91c64697f4b22241f1c25d4637dc494f
Aug 12 03:53:19 dist-test-slave-tm1g krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506798, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:53:19.035815 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.024212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36155 (local address 127.17.54.194:55349)
0812 03:53:19.024494 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:19.024548 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:19.025194 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:19.025481 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:19.025485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:19.026355 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:19.026609 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.026615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.027447 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.027448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.028371 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.028373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.028839 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.028840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.029040 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.029649 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:19.029658 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:19.031615 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:19.033952 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:19.033962 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:19.033965 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:19.034243 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:19.034698 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:19.034699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:19.034700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:19.034832 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:19.035213 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:19.035220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:19.035375 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 03:53:19.035476 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:53:19.035622 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
I20260812 03:53:19.035807 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.024264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:55349 (local address 127.17.54.254:36155)
0812 03:53:19.024375 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.024378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.024655 (+ 277us) server_negotiation.cc:407] Connection header received
0812 03:53:19.025336 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.025338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.025353 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.025408 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:19.026744 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.027326 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.028489 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.028718 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.028824 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.031732 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:19.031734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:19.031734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:53:19.031743 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:19.033835 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:19.034510 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:19.034512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:19.034512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:19.034584 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:19.034926 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:19.034928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:19.034928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:19.035100 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:19.035174 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.035545 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.035660 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:53:19.036942 29740 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36155
I20260812 03:53:19.037088 29740 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:19.037336 29740 heartbeater.cc:511] Master 127.17.54.254:36155 requested a full tablet report, sending...
I20260812 03:53:19.037815 29351 ts_manager.cc:194] Registered new tserver with Master: ce1c4deea82744e5adbd247719db25c4 (127.17.54.194:39991)
I20260812 03:53:19.038234 29351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:55349
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:53:19.062026 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36155
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:53:19.148056 29745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:53:19.148329 29745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:53:19.148376 29745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:53:19.149605 29745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:53:19.149688 29745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:53:19.149737 29745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:53:19.151443 29745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:53:19.151772 29745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:53:19.152035 29745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:53:19.157608 29753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:19.157601 29750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:53:19.157601 29751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:53:19.158146 29745 server_base.cc:1034] running on GCE node
I20260812 03:53:19.158420 29745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:53:19.158705 29745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:53:19.159864 29745 hybrid_clock.cc:648] HybridClock initialized: now 1786506799159850 us; error 31 us; skew 500 ppm
Aug 12 03:53:19 dist-test-slave-tm1g krb5kdc[29293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:53:19.162568 29745 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:53:19.163043 29745 webserver.cc:460] Webserver started at http://127.17.54.195:39465/ using document root <none> and password file <none>
I20260812 03:53:19.163288 29745 fs_manager.cc:359] Metadata directory not provided
I20260812 03:53:19.163363 29745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:53:19.163529 29745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:53:19.164624 29745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6"
format_stamp: "Formatted at 2026-08-12 03:53:19 on dist-test-slave-tm1g"
server_key: "23521c0254e4e5c5ca9eaf870ceb44dc"
server_key_iv: "204ed216b919949091bb8b02c875106a"
server_key_version: "encryptionkey@0"
I20260812 03:53:19.165021 29745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6"
format_stamp: "Formatted at 2026-08-12 03:53:19 on dist-test-slave-tm1g"
server_key: "23521c0254e4e5c5ca9eaf870ceb44dc"
server_key_iv: "204ed216b919949091bb8b02c875106a"
server_key_version: "encryptionkey@0"
I20260812 03:53:19.166653 29745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 03:53:19.167578 29760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:19.167799 29745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:53:19.167886 29745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6"
format_stamp: "Formatted at 2026-08-12 03:53:19 on dist-test-slave-tm1g"
server_key: "23521c0254e4e5c5ca9eaf870ceb44dc"
server_key_iv: "204ed216b919949091bb8b02c875106a"
server_key_version: "encryptionkey@0"
I20260812 03:53:19.167949 29745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:53:19.179282 29745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:53:19.181710 29745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:53:19.181828 29745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:53:19.182019 29745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:53:19.182437 29745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:53:19.182479 29745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:19.182528 29745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:53:19.182555 29745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:53:19.188966 29745 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:43791
I20260812 03:53:19.189041 29896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:43791 every 8 connection(s)
I20260812 03:53:19.189415 29745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:53:19 dist-test-slave-tm1g krb5kdc[29293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:53:19.198369 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 29745
I20260812 03:53:19.198470 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:53:19.198680 17627 external_mini_cluster.cc:1428] Setting key 097836287ececfefe0b485ad26c16ef6
I20260812 03:53:19.200008 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.190269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:34653 (local address 127.17.54.254:36155)
0812 03:53:19.190359 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.190361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.190618 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:53:19.191183 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.191185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.191199 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.191253 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:19.192264 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.192795 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.193850 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.194072 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.194155 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.196645 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:19.196646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:19.196647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:53:19.196655 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:19.198080 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:19.198668 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:19.198669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:19.198670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:19.198763 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:19.199169 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:19.199171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:19.199171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:19.199415 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:19.199486 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.199865 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.199941 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:53:19.200223 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.190243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36155 (local address 127.17.54.195:34653)
0812 03:53:19.190485 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:19.190523 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:19.191059 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:19.191326 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:19.191329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:19.191920 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:53:19.192150 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.192155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.192905 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.192906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.193732 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.193735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.194175 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.194176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.194357 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.194952 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:53:19.194960 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:53:19.196527 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:53:19.198181 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:19.198190 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:19.198193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:19.198590 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:19.198883 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:53:19.198884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:53:19.198885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:53:19.199050 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:53:19.199494 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:53:19.199501 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:53:19.199694 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 03:53:19.199803 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:53:19.199957 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":139,"threads_started":1}
I20260812 03:53:19.200727 29897 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36155
I20260812 03:53:19.200842 29897 heartbeater.cc:464] Registering TS with master...
I20260812 03:53:19.201040 29897 heartbeater.cc:511] Master 127.17.54.254:36155 requested a full tablet report, sending...
I20260812 03:53:19.201462 29351 ts_manager.cc:194] Registered new tserver with Master: 1e8720312b4f4e80ae3bb7ba0ca3cff6 (127.17.54.195:43791)
I20260812 03:53:19.201896 29351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:34653
I20260812 03:53:19.210988 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:53:19.219497 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.212565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.17.54.254:36155)
0812 03:53:19.212649 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.212651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.212660 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:53:19.212690 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.212691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.212699 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.212781 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:53:19.213407 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.213919 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.214796 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.214995 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.215115 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.216411 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:53:19.216412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:53:19.216413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:53:19.216421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:53:19.217891 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:19.218478 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:19.218480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:19.218480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:19.218555 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:53:19.218802 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:53:19.218803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:53:19.218803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:53:19.218992 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:53:19.219116 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.219279 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.219377 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:53:19.221271 29351 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:53:19.223166 29351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:53:19.230729 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.226498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.17.54.195:43791)
0812 03:53:19.226722 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.226727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.226739 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:53:19.226854 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.226857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.226876 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.226970 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:53:19.227224 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.227956 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.229395 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.230201 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.230276 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.230361 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.230567 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.230618 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260812 03:53:19.230782 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.226329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:39991 (local address 127.0.0.1:39828)
0812 03:53:19.226463 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:19.226471 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:19.226513 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:19.227105 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:19.227106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:19.227109 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:53:19.227185 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.227187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.228588 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.228590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.229713 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.229716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.230562 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.230563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.230615 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.230619 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:53:19.230647 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 03:53:19.230680 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260812 03:53:19.230697 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.226059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:43791 (local address 127.0.0.1:58454)
0812 03:53:19.226262 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:19.226273 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:19.226363 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:19.227074 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:19.227078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:19.227080 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:53:19.227158 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.227161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.228072 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.228073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.229286 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.229289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.230328 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.230330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.230440 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.230458 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 03:53:19.230532 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:53:19.230597 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260812 03:53:19.231009 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.226719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.17.54.194:39991)
0812 03:53:19.226928 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.226933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.226942 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:53:19.226979 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.226981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.226996 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.227101 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:53:19.227276 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.228446 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.229822 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.230458 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.230741 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.230824 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.230854 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.230905 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
I20260812 03:53:19.231534 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.226056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:42221 (local address 127.0.0.1:54678)
0812 03:53:19.226222 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:19.226266 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:19.226363 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:19.227154 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:19.227155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:19.227156 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:53:19.227191 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.227192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.228570 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.228571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.229731 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.229733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.230705 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.230706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.230777 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.230781 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:53:19.231414 (+ 633us) client_negotiation.cc:240] Negotiation successful
0812 03:53:19.231463 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":34,"thread_start_us":76,"threads_started":1}
I20260812 03:53:19.231729 29817 tablet_service.cc:1467] Processing CreateTablet for tablet 117b63954d2148bba9bcddc342ed09ec (DEFAULT_TABLE table=test-table [id=0aec54d6fc7949df978fed8570faf140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:53:19.231935 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.226166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.17.54.193:42221)
0812 03:53:19.226962 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.226966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.226978 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:53:19.227031 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.227034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.227050 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.227267 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:53:19.227348 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.228444 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.229837 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.230604 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.231705 (+ 1101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.231773 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.231800 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.231849 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":93,"threads_started":1}
I20260812 03:53:19.231813 29662 tablet_service.cc:1467] Processing CreateTablet for tablet 117b63954d2148bba9bcddc342ed09ec (DEFAULT_TABLE table=test-table [id=0aec54d6fc7949df978fed8570faf140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:53:19.232158 29817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117b63954d2148bba9bcddc342ed09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:53:19.232190 29662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117b63954d2148bba9bcddc342ed09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:53:19.232570 29502 tablet_service.cc:1467] Processing CreateTablet for tablet 117b63954d2148bba9bcddc342ed09ec (DEFAULT_TABLE table=test-table [id=0aec54d6fc7949df978fed8570faf140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:53:19.232880 29502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117b63954d2148bba9bcddc342ed09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:53:19.235231 29918 tablet_bootstrap.cc:492] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: Bootstrap starting.
I20260812 03:53:19.235646 29920 tablet_bootstrap.cc:492] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Bootstrap starting.
I20260812 03:53:19.236100 29918 tablet_bootstrap.cc:654] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: Neither blocks nor log segments found. Creating new log.
I20260812 03:53:19.236274 29921 tablet_bootstrap.cc:492] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9: Bootstrap starting.
I20260812 03:53:19.236320 29920 tablet_bootstrap.cc:654] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Neither blocks nor log segments found. Creating new log.
I20260812 03:53:19.236856 29921 tablet_bootstrap.cc:654] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9: Neither blocks nor log segments found. Creating new log.
I20260812 03:53:19.236856 29920 log.cc:824] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:53:19.237447 29918 log.cc:824] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:53:19.237458 29921 log.cc:824] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:53:19.238366 29920 tablet_bootstrap.cc:492] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: No bootstrap required, opened a new log
I20260812 03:53:19.238430 29918 tablet_bootstrap.cc:492] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: No bootstrap required, opened a new log
I20260812 03:53:19.238461 29920 ts_tablet_manager.cc:1397] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:53:19.238515 29918 ts_tablet_manager.cc:1397] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:53:19.238600 29921 tablet_bootstrap.cc:492] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9: No bootstrap required, opened a new log
I20260812 03:53:19.238667 29921 ts_tablet_manager.cc:1397] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:53:19.239948 29918 raft_consensus.cc:348] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.240119 29918 raft_consensus.cc:374] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:53:19.240165 29918 raft_consensus.cc:729] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e8720312b4f4e80ae3bb7ba0ca3cff6, State: Initialized, Role: FOLLOWER
I20260812 03:53:19.240284 29918 consensus_queue.cc:260] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.240600 29897 heartbeater.cc:503] Master 127.17.54.254:36155 was elected leader, sending a full tablet report...
I20260812 03:53:19.240613 29918 ts_tablet_manager.cc:1428] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:53:19.240528 29920 raft_consensus.cc:348] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.240698 29920 raft_consensus.cc:374] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:53:19.240633 29921 raft_consensus.cc:348] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.240732 29920 raft_consensus.cc:729] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1c4deea82744e5adbd247719db25c4, State: Initialized, Role: FOLLOWER
I20260812 03:53:19.240767 29921 raft_consensus.cc:374] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:53:19.240798 29921 raft_consensus.cc:729] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f65cc0722846d68f7af52747b758f9, State: Initialized, Role: FOLLOWER
I20260812 03:53:19.240852 29920 consensus_queue.cc:260] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.240897 29921 consensus_queue.cc:260] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.241146 29920 ts_tablet_manager.cc:1428] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:53:19.241153 29921 ts_tablet_manager.cc:1428] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 03:53:19.241268 29740 heartbeater.cc:503] Master 127.17.54.254:36155 was elected leader, sending a full tablet report...
I20260812 03:53:19.241282 29584 heartbeater.cc:503] Master 127.17.54.254:36155 was elected leader, sending a full tablet report...
I20260812 03:53:19.261817 29929 raft_consensus.cc:482] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:53:19.262005 29929 raft_consensus.cc:504] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.262502 29929 leader_election.cc:288] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e8720312b4f4e80ae3bb7ba0ca3cff6 (127.17.54.195:43791), ce1c4deea82744e5adbd247719db25c4 (127.17.54.194:39991)
I20260812 03:53:19.265856 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.263085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:57631 (local address 127.17.54.194:39991)
0812 03:53:19.263138 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.263140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.263153 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:53:19.263224 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.263225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.263235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.263280 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:53:19.263450 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.263968 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.265048 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.265465 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.265558 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.265567 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.265739 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.265779 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:53:19.265862 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.263067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:55573 (local address 127.17.54.195:43791)
0812 03:53:19.263140 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.263143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.263155 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:53:19.263184 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.263184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.263192 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.263263 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:53:19.263484 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.263966 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.265052 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.265481 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.265580 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.265588 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.265749 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.265788 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:53:19.265878 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.262734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:43791 (local address 127.17.54.193:55573)
0812 03:53:19.262831 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:19.262854 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:19.262954 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:19.263272 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:19.263273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:19.263275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:53:19.263366 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.263368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.264072 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.264073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.264943 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.264945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.265580 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.265581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.265667 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.265670 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:53:19.265742 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:53:19.265795 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":34}
I20260812 03:53:19.265887 29930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.262883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:39991 (local address 127.17.54.193:57631)
0812 03:53:19.263105 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:19.263115 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:53:19.263168 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:53:19.263296 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:53:19.263297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:53:19.263298 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:53:19.263369 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.263371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.264062 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.264063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.264943 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:53:19.264946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.265571 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.265573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:53:19.265646 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.265650 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:53:19.265720 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:53:19.265768 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":49,"thread_start_us":134,"threads_started":1}
I20260812 03:53:19.266240 29838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117b63954d2148bba9bcddc342ed09ec" candidate_uuid: "68f65cc0722846d68f7af52747b758f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" is_pre_election: true
I20260812 03:53:19.266404 29685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117b63954d2148bba9bcddc342ed09ec" candidate_uuid: "68f65cc0722846d68f7af52747b758f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1c4deea82744e5adbd247719db25c4" is_pre_election: true
I20260812 03:53:19.266464 29838 raft_consensus.cc:2454] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f65cc0722846d68f7af52747b758f9 in term 0.
I20260812 03:53:19.266547 29685 raft_consensus.cc:2454] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f65cc0722846d68f7af52747b758f9 in term 0.
I20260812 03:53:19.266719 29444 leader_election.cc:302] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e8720312b4f4e80ae3bb7ba0ca3cff6, 68f65cc0722846d68f7af52747b758f9; no voters:
I20260812 03:53:19.266863 29929 raft_consensus.cc:2781] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:53:19.266978 29929 raft_consensus.cc:482] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:53:19.267021 29929 raft_consensus.cc:3037] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:53:19.267789 29929 raft_consensus.cc:504] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.267956 29929 leader_election.cc:288] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [CANDIDATE]: Term 1 election: Requested vote from peers 1e8720312b4f4e80ae3bb7ba0ca3cff6 (127.17.54.195:43791), ce1c4deea82744e5adbd247719db25c4 (127.17.54.194:39991)
I20260812 03:53:19.268214 29838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117b63954d2148bba9bcddc342ed09ec" candidate_uuid: "68f65cc0722846d68f7af52747b758f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6"
I20260812 03:53:19.268301 29838 raft_consensus.cc:3037] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:53:19.268260 29685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117b63954d2148bba9bcddc342ed09ec" candidate_uuid: "68f65cc0722846d68f7af52747b758f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1c4deea82744e5adbd247719db25c4"
I20260812 03:53:19.268378 29685 raft_consensus.cc:3037] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:53:19.269186 29838 raft_consensus.cc:2454] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f65cc0722846d68f7af52747b758f9 in term 1.
I20260812 03:53:19.269191 29685 raft_consensus.cc:2454] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f65cc0722846d68f7af52747b758f9 in term 1.
I20260812 03:53:19.269368 29444 leader_election.cc:302] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f65cc0722846d68f7af52747b758f9, ce1c4deea82744e5adbd247719db25c4; no voters:
I20260812 03:53:19.269483 29929 raft_consensus.cc:2781] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:53:19.269656 29929 raft_consensus.cc:686] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [term 1 LEADER]: Becoming Leader. State: Replica: 68f65cc0722846d68f7af52747b758f9, State: Running, Role: LEADER
I20260812 03:53:19.269739 29929 consensus_queue.cc:237] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } }
I20260812 03:53:19.270720 29350 catalog_manager.cc:5495] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68f65cc0722846d68f7af52747b758f9 (127.17.54.193). New cstate: current_term: 1 leader_uuid: "68f65cc0722846d68f7af52747b758f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f65cc0722846d68f7af52747b758f9" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 42221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 } health_report { overall_health: UNKNOWN } } }
W20260812 03:53:19.274102 29741 tablet.cc:2369] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:53:19.277072 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.274183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.17.54.193:42221)
0812 03:53:19.274256 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:53:19.274258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:53:19.274268 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:53:19.274421 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:53:19.274422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:53:19.274448 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:53:19.274532 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:53:19.274699 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.275253 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.276148 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:53:19.276385 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:53:19.276475 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:53:19.276669 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:53:19.276787 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:53:19.276902 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:53:19.276925 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 03:53:19.276948 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:53:19.279325 29838 raft_consensus.cc:1261] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 1 FOLLOWER]: Refusing update from remote peer 68f65cc0722846d68f7af52747b758f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:53:19.279366 29685 raft_consensus.cc:1261] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 1 FOLLOWER]: Refusing update from remote peer 68f65cc0722846d68f7af52747b758f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:53:19.279716 29931 consensus_queue.cc:1035] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e8720312b4f4e80ae3bb7ba0ca3cff6" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 43791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:53:19.279847 29929 consensus_queue.cc:1035] T 117b63954d2148bba9bcddc342ed09ec P 68f65cc0722846d68f7af52747b758f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1c4deea82744e5adbd247719db25c4" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:53:19.282742 29934 mvcc.cc:205] Tried to move back new op lower bound from 7317531849844244480 to 7317531849809739776. Current Snapshot: MvccSnapshot[applied={T|T < 7317531849844244480 or (T in {7317531849844244480})}]
I20260812 03:53:19.283401 29935 mvcc.cc:205] Tried to move back new op lower bound from 7317531849844244480 to 7317531849809739776. Current Snapshot: MvccSnapshot[applied={T|T < 7317531849844244480}]
I20260812 03:53:19.284658 29351 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:53:19.284761 29351 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:53:19.285563 29351 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a193dc53ce3345858459e5faeacc3d22: Sending DeleteTablet for 3 replicas of tablet 117b63954d2148bba9bcddc342ed09ec
I20260812 03:53:19.285894 29817 tablet_service.cc:1514] Processing DeleteTablet for tablet 117b63954d2148bba9bcddc342ed09ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:53:19 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:58454
I20260812 03:53:19.285897 29502 tablet_service.cc:1514] Processing DeleteTablet for tablet 117b63954d2148bba9bcddc342ed09ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:53:19 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:54678
I20260812 03:53:19.285939 29662 tablet_service.cc:1514] Processing DeleteTablet for tablet 117b63954d2148bba9bcddc342ed09ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:53:19 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:39828
I20260812 03:53:19.286397 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 29425
I20260812 03:53:19.286557 29947 tablet_replica.cc:330] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: stopping tablet replica
I20260812 03:53:19.286659 29947 raft_consensus.cc:2229] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:53:19.286752 29947 raft_consensus.cc:2258] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:53:19.288633 29947 ts_tablet_manager.cc:1905] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:53:19.290622 29947 ts_tablet_manager.cc:1918] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:53:19.290704 29947 log.cc:1196] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/117b63954d2148bba9bcddc342ed09ec
I20260812 03:53:19.290997 29947 ts_tablet_manager.cc:1939] T 117b63954d2148bba9bcddc342ed09ec P 1e8720312b4f4e80ae3bb7ba0ca3cff6: Deleting consensus metadata
I20260812 03:53:19.291829 29331 catalog_manager.cc:4841] TS 1e8720312b4f4e80ae3bb7ba0ca3cff6 (127.17.54.195:43791): tablet 117b63954d2148bba9bcddc342ed09ec (table test-table [id=0aec54d6fc7949df978fed8570faf140]) successfully deleted
I20260812 03:53:19.298755 29948 tablet_replica.cc:330] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: stopping tablet replica
I20260812 03:53:19.298954 29948 raft_consensus.cc:2229] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:53:19.299090 29948 raft_consensus.cc:2258] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:53:19.301777 29948 ts_tablet_manager.cc:1905] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:53:19.304451 29948 ts_tablet_manager.cc:1918] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:53:19.304559 29948 log.cc:1196] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/117b63954d2148bba9bcddc342ed09ec
I20260812 03:53:19.304880 29948 ts_tablet_manager.cc:1939] T 117b63954d2148bba9bcddc342ed09ec P ce1c4deea82744e5adbd247719db25c4: Deleting consensus metadata
I20260812 03:53:19.305473 29331 catalog_manager.cc:4841] TS ce1c4deea82744e5adbd247719db25c4 (127.17.54.194:39991): tablet 117b63954d2148bba9bcddc342ed09ec (table test-table [id=0aec54d6fc7949df978fed8570faf140]) successfully deleted
W20260812 03:53:19.305557 29330 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:42221 (error 108)
I20260812 03:53:19.306177 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 29591
I20260812 03:53:19.306391 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:53:19.306163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:42221 (local address 127.0.0.1:54694)
0812 03:53:19.306230 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 03:53:19.306304 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 03:53:19.309170 29330 catalog_manager.cc:4584] TS 68f65cc0722846d68f7af52747b758f9 (127.17.54.193:42221): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 117b63954d2148bba9bcddc342ed09ec: Network error: Client connection negotiation failed: client connection to 127.17.54.193:42221: connect: Connection refused (error 111)
I20260812 03:53:19.315258 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 29745
I20260812 03:53:19.322217 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 29313
2026-08-12T03:53:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2101 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:53:19 dist-test-slave-tm1g krb5kdc[29954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:53:19 dist-test-slave-tm1g krb5kdc[29954](info): set up 2 sockets
Aug 12 03:53:19 dist-test-slave-tm1g krb5kdc[29954](info): commencing operation
krb5kdc: starting...
W20260812 03:53:20.498520 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:53:20 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:53:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:53:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.54.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.54.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/rangeradmin_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/rangeradmin_127.17.54.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.54.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.54.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/rangerlookup_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/rangerlookup_127.17.54.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.54.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.54.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/HTTP_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/HTTP_127.17.54.212@KRBTEST.COM.keytab.
I20260812 03:53:20.703341 17627 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task75kCRI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/postgres -l logfile start
2026-08-12 03:53:23.871 UTC [29990] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:53:23.871 UTC [29990] LOG: listening on IPv4 address "127.17.54.212", port 38201
2026-08-12 03:53:23.876 UTC [29990] LOG: listening on Unix socket "/tmp/.s.PGSQL.38201"
2026-08-12 03:53:23.899 UTC [29994] LOG: database system was shut down at 2026-08-12 03:53:21 UTC
2026-08-12 03:53:23.904 UTC [29990] LOG: database system is ready to accept connections
I20260812 03:53:24.940619 17627 mini_postgres.cc:96] Postgres bound to 38201
2026-08-12 03:53:24.948 UTC [30004] FATAL: database "slave" does not exist
127.17.54.212:38201 - accepting connections
I20260812 03:53:24.948853 17627 mini_ranger.cc:161] Starting Ranger...
I20260812 03:53:24.965215 17627 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:53:25.594661 17627 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:53:25.594735 17627 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-admin
2026-08-12 03:53:25,899 [I] DB FLAVOR :POSTGRES
2026-08-12 03:53:25,900 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:53:25,900 [I] Checking connection..
2026-08-12 03:53:25,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:53:26,173 [I] Checking connection passed.
2026-08-12 03:53:26,173 [I] --------- Verifying version history table ---------
2026-08-12 03:53:26,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:53:26,449 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:53:26,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:53:26,706 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:53:26,707 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:53:26,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:53:26.983 UTC [30084] WARNING: there is no transaction in progress
2026-08-12 03:53:26,991 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:53:26,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:53:27,261 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:53:27,261 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:53:27,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:53:27,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:53:27,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tm1g.c.gcp-upstream.internal','N') ;"
2026-08-12 03:53:28,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:53:28,326 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:53:28,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:53:28,592 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:53:28,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:53:28,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:53:29,182 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:53:29,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:53:30.281 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.300 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.320 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.340 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.359 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.379 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.443 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.451 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.469 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.480 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.493 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.510 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.521 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.529 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.538 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:30.545 UTC [30204] WARNING: there is no transaction in progress
2026-08-12 03:53:31,297 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:53:31,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:53:31,562 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:53:31,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:53:31,841 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:53:31,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:53:32,113 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:53:32,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:53:32,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:53:32,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tm1g.c.gcp-upstream.internal';"
2026-08-12 03:53:32,932 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:53:32,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:53:33,207 [I] DB_PATCHES have already been applied
I20260812 03:53:33.213866 17627 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-admin:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task75kCRI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:53:33.213943 17627 mini_ranger.cc:193] Using host: 127.17.54.212
I20260812 03:53:33.217871 17627 mini_ranger.cc:227] Ranger admin URL: http://127.17.54.212:40877
Aug 12, 2026 3:53:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:53:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:53:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:53:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:53:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:53:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:53:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:53:34.442040 17627 mini_ranger.cc:160] Time spent starting Ranger: real 9.493s user 0.000s sys 0.012s
Aug 12, 2026 3:53:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:53:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 3:53:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:53:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:53:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:53:57.632432 17627 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.54.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.54.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/rangerkms_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/rangerkms_127.17.54.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.54.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.54.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.54.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/HTTP_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.54.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/HTTP_127.17.54.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:53:57 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506837, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:53:57.760005 17627 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:53:57.774904 17627 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:53:59.117825 17627 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:53:59.117926 17627 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms
I20260812 03:53:59.117944 17627 mini_ranger_kms.cc:227] Using postgres at 127.17.54.212:38201
2026-08-12 03:53:59,604 [I] DB FLAVOR :POSTGRES
2026-08-12 03:53:59,604 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:53:59,604 [I] Checking connection
2026-08-12 03:53:59,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:53:59,981 [I] connection success
2026-08-12 03:53:59,982 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:53:59,982 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:53:59,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:54:00,286 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:54:00,286 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:54:00,287 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:54:00,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:38201/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:54:00,567 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:54:01.116604 17627 mini_ranger_kms.cc:310] Created kms service
I20260812 03:54:01.327152 17627 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:54:01.379084 17627 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:54:01 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506837, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:54:01.842762 17627 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:54:01.842916 17627 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms:/tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:54:01.842939 17627 mini_ranger_kms.cc:240] Using host: 127.17.54.212
I20260812 03:54:01.847481 17627 mini_ranger_kms.cc:276] Ranger KMS PID: 30420
I20260812 03:54:01.847647 17627 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.54.212:33021
Aug 12, 2026 3:54:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:54:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:54:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:54:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:54:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:54:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 03:54:04.279487 17627 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.519s user 0.007s sys 0.443s
I20260812 03:54:04.279604 17627 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:54:04.279688 17627 mini_ranger_kms.cc:401] 127.17.54.212:33021/kms/v1/keys
Aug 12 03:54:04 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.54.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 03:54:04 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.54.212@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
Aug 12 03:54:05 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.54.212@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
Aug 12 03:54:07 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:54:07.918555 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33757
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:33757
--ranger_config_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:33021/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:54:08.017580 30481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:54:08.017855 30481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:54:08.017899 30481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:54:08.019191 30481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:54:08.019268 30481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:54:08.019294 30481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:54:08.019316 30481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:54:08.019344 30481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:54:08.021189 30481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.54.212:33021/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:33757
--ranger_config_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:54:08.021522 30481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:54:08.021777 30481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:54:08.028995 30487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:54:08.029183 30481 server_base.cc:1034] running on GCE node
W20260812 03:54:08.029202 30486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:54:08.029027 30490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:54:08.029762 30481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:54:08.030146 30481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:54:08.031324 30481 hybrid_clock.cc:648] HybridClock initialized: now 1786506848031300 us; error 50 us; skew 500 ppm
Aug 12 03:54:08 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506848, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:54:08.038223 30481 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:54:08.038811 30481 webserver.cc:460] Webserver started at http://127.17.54.254:44919/ using document root <none> and password file <none>
I20260812 03:54:08.039075 30481 fs_manager.cc:359] Metadata directory not provided
I20260812 03:54:08.039143 30481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:54:08.039307 30481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:54:08 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506848, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:54:08.092258 30481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "cc5692ea8ce547a68726cdd6354d2c81"
format_stamp: "Formatted at 2026-08-12 03:54:08 on dist-test-slave-tm1g"
server_key: "e8866ca7e8642116af43298b740bbf6d"
server_key_iv: "a1279fd6123204184702e5e7c22d75c0"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:08.092756 30481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "cc5692ea8ce547a68726cdd6354d2c81"
format_stamp: "Formatted at 2026-08-12 03:54:08 on dist-test-slave-tm1g"
server_key: "e8866ca7e8642116af43298b740bbf6d"
server_key_iv: "a1279fd6123204184702e5e7c22d75c0"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:08.094487 30481 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:54:08.107578 30497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:08.107839 30481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:54:08.107962 30481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "cc5692ea8ce547a68726cdd6354d2c81"
format_stamp: "Formatted at 2026-08-12 03:54:08 on dist-test-slave-tm1g"
server_key: "e8866ca7e8642116af43298b740bbf6d"
server_key_iv: "a1279fd6123204184702e5e7c22d75c0"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:08.108089 30481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:54:08.131749 30481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:54:08.134011 30481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:54:08.134245 30481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:54:08.142613 30481 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:33757
I20260812 03:54:08.142643 30557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:33757 every 8 connection(s)
I20260812 03:54:08.143184 30481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:54:08.144135 30559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:54:08.146193 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 30481
I20260812 03:54:08.146322 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:54:08.146587 17627 external_mini_cluster.cc:1428] Setting key c2ac468dc24e0b3c856903a15e219547
I20260812 03:54:08.147240 30559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81: Bootstrap starting.
I20260812 03:54:08.148151 30559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81: Neither blocks nor log segments found. Creating new log.
I20260812 03:54:08.148772 30559 log.cc:824] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81: Log is configured to *not* fsync() on all Append() calls
I20260812 03:54:08.149735 30559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81: No bootstrap required, opened a new log
I20260812 03:54:08.152822 30559 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5692ea8ce547a68726cdd6354d2c81" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33757 } }
I20260812 03:54:08.153007 30559 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:54:08.153040 30559 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5692ea8ce547a68726cdd6354d2c81, State: Initialized, Role: FOLLOWER
I20260812 03:54:08.153245 30559 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5692ea8ce547a68726cdd6354d2c81" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33757 } }
I20260812 03:54:08.153350 30559 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:54:08.153417 30559 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:54:08.153478 30559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:54:08.154270 30559 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5692ea8ce547a68726cdd6354d2c81" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33757 } }
I20260812 03:54:08.154443 30559 leader_election.cc:302] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc5692ea8ce547a68726cdd6354d2c81; no voters:
I20260812 03:54:08.154688 30559 leader_election.cc:288] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:54:08.154804 30566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:54:08.155145 30566 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [term 1 LEADER]: Becoming Leader. State: Replica: cc5692ea8ce547a68726cdd6354d2c81, State: Running, Role: LEADER
I20260812 03:54:08.155171 30559 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:54:08.156275 30566 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5692ea8ce547a68726cdd6354d2c81" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33757 } }
Aug 12 03:54:08 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:54:08.157950 30567 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc5692ea8ce547a68726cdd6354d2c81. Latest consensus state: current_term: 1 leader_uuid: "cc5692ea8ce547a68726cdd6354d2c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5692ea8ce547a68726cdd6354d2c81" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33757 } } }
I20260812 03:54:08.158056 30567 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [sys.catalog]: This master's current role is: LEADER
I20260812 03:54:08.158239 30559 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:54:08.158373 30568 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc5692ea8ce547a68726cdd6354d2c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5692ea8ce547a68726cdd6354d2c81" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33757 } } }
I20260812 03:54:08.158440 30568 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81 [sys.catalog]: This master's current role is: LEADER
I20260812 03:54:08.158856 30581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:54:08.164933 30581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:54:08.168061 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:08.147195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.17.54.254:33757)
0812 03:54:08.148415 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:08.148425 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:08.148489 (+ 64us) server_negotiation.cc:407] Connection header received
0812 03:54:08.149248 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:08.149258 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:08.149327 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:08.149574 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:54:08.150602 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:08.152138 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:08.153322 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:08.154087 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:08.154206 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:08.157356 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:54:08.157361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:54:08.157362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:54:08.157376 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:54:08.163813 (+ 6437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:08.166020 (+ 2207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:08.166025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:08.166025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:08.166175 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:08.166559 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:08.166560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:08.166561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:08.167017 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:54:08.167232 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:08.167642 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 03:54:08.167815 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1146,"thread_start_us":134,"threads_started":1}
I20260812 03:54:08.170907 30581 catalog_manager.cc:1347] Generated new cluster ID: e53ce80641424a74bb3806230ba72969
I20260812 03:54:08.170994 30581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:54:08.191011 30581 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:54:08.192163 30581 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:54:08.214915 30581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81: Generated new TSK 0
I20260812 03:54:08.215276 30581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:54:09.107151 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:33021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33757
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:54:09.273427 30613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:54:09.273972 30613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:54:09.274075 30613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:54:09.276145 30613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:54:09.276294 30613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:54:09.276418 30613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:54:09.279428 30613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.54.212:33021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:54:09.280041 30613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:54:09.280485 30613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:54:09.289661 30623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:54:09.291275 30621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:54:09.291458 30620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:54:09.292469 30613 server_base.cc:1034] running on GCE node
I20260812 03:54:09.292728 30613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:54:09.293126 30613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:54:09.294319 30613 hybrid_clock.cc:648] HybridClock initialized: now 1786506849294276 us; error 38 us; skew 500 ppm
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:54:09.298804 30613 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:54:09.299599 30613 webserver.cc:460] Webserver started at http://127.17.54.193:37081/ using document root <none> and password file <none>
I20260812 03:54:09.299995 30613 fs_manager.cc:359] Metadata directory not provided
I20260812 03:54:09.300173 30613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:54:09.300411 30613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:54:09.320497 30613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "1c53be68db3f49a3afa05d24368e78ce"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "974db7a9b804a0e5a72c972435f86511"
server_key_iv: "f8549fc475bf7184d93d1608981ebaba"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.320986 30613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "1c53be68db3f49a3afa05d24368e78ce"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "974db7a9b804a0e5a72c972435f86511"
server_key_iv: "f8549fc475bf7184d93d1608981ebaba"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.326759 30613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260812 03:54:09.340049 30631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.340238 30613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:54:09.340327 30613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "1c53be68db3f49a3afa05d24368e78ce"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "974db7a9b804a0e5a72c972435f86511"
server_key_iv: "f8549fc475bf7184d93d1608981ebaba"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.340443 30613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:54:09.358507 30613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:54:09.361102 30613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:54:09.361279 30613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:54:09.361563 30613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:54:09.361989 30613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:54:09.362049 30613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.362094 30613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:54:09.362119 30613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.375651 30613 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:38983
I20260812 03:54:09.376215 30613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:54:09.376730 30767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:38983 every 8 connection(s)
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:54:09.383769 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 30613
I20260812 03:54:09.383874 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:54:09.384099 17627 external_mini_cluster.cc:1428] Setting key bd679d83922e8acf8d06bd0e1fd24f3b
I20260812 03:54:09.391665 30771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:09.377508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33757 (local address 127.17.54.193:59007)
0812 03:54:09.377856 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:09.377893 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:54:09.378658 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:54:09.378956 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:54:09.378960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:54:09.379262 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:54:09.379582 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:09.379587 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.380332 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.380333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:09.381275 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:09.381278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.381681 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.381682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:09.381835 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:09.382455 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:54:09.382463 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:54:09.383996 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:54:09.389667 (+ 5671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:54:09.389677 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:54:09.389679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:54:09.390036 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:54:09.390361 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:54:09.390362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:54:09.390362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:54:09.390517 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:54:09.390907 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:54:09.390915 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:54:09.391097 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 03:54:09.391200 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 03:54:09.391380 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":103,"threads_started":1}
I20260812 03:54:09.392107 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:09.377641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:59007 (local address 127.17.54.254:33757)
0812 03:54:09.377749 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:09.377751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:09.377994 (+ 243us) server_negotiation.cc:407] Connection header received
0812 03:54:09.378772 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:09.378774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:09.378788 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:09.378871 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:54:09.379686 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.380228 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.381371 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.381596 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.381674 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:09.384109 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:54:09.384111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:54:09.384111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:54:09.384129 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:54:09.389551 (+ 5422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:09.390147 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:09.390149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:09.390149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:09.390224 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:09.390639 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:09.390640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:09.390640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:09.390797 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:54:09.391823 (+ 1026us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:09.391897 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:54:09.391995 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":47,"threads_started":1}
I20260812 03:54:09.392580 30769 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33757
I20260812 03:54:09.392696 30769 heartbeater.cc:464] Registering TS with master...
I20260812 03:54:09.392935 30769 heartbeater.cc:511] Master 127.17.54.254:33757 requested a full tablet report, sending...
I20260812 03:54:09.393514 30515 ts_manager.cc:194] Registered new tserver with Master: 1c53be68db3f49a3afa05d24368e78ce (127.17.54.193:38983)
I20260812 03:54:09.394466 30515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:59007
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:54:09.422488 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:33021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33757
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:54:09.564013 30776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:54:09.564307 30776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:54:09.564345 30776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:54:09.566217 30776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:54:09.566293 30776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:54:09.566365 30776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:54:09.569036 30776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.54.212:33021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:54:09.569321 30776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:54:09.569659 30776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:54:09.577772 30785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:54:09.578377 30788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:54:09.578517 30786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:54:09.579780 30776 server_base.cc:1034] running on GCE node
I20260812 03:54:09.580134 30776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:54:09.580451 30776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:54:09.581671 30776 hybrid_clock.cc:648] HybridClock initialized: now 1786506849581637 us; error 53 us; skew 500 ppm
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:54:09.585755 30776 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:54:09.586362 30776 webserver.cc:460] Webserver started at http://127.17.54.194:39901/ using document root <none> and password file <none>
I20260812 03:54:09.586722 30776 fs_manager.cc:359] Metadata directory not provided
I20260812 03:54:09.586804 30776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:54:09.586993 30776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:54:09.604477 30776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "99daf5feda3647f0a509c11ff5a8555f"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "850aced67210daff84a7b1e102aad1d5"
server_key_iv: "19bdd8f1eb5228c43f76fa2e59774e4e"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.605103 30776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "99daf5feda3647f0a509c11ff5a8555f"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "850aced67210daff84a7b1e102aad1d5"
server_key_iv: "19bdd8f1eb5228c43f76fa2e59774e4e"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.621011 30776 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.002s sys 0.000s
I20260812 03:54:09.637612 30799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.637879 30776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 03:54:09.637989 30776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "99daf5feda3647f0a509c11ff5a8555f"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "850aced67210daff84a7b1e102aad1d5"
server_key_iv: "19bdd8f1eb5228c43f76fa2e59774e4e"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.638094 30776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:54:09.668866 30776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:54:09.671620 30776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:54:09.671868 30776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:54:09.672230 30776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:54:09.672739 30776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:54:09.672896 30776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.673039 30776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:54:09.673205 30776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.687968 30776 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:43543
I20260812 03:54:09.688094 30927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:43543 every 8 connection(s)
I20260812 03:54:09.688701 30776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:54:09.696904 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 30776
I20260812 03:54:09.697058 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:54:09.697280 17627 external_mini_cluster.cc:1428] Setting key af20e4fc583af0d5ae8d9bcb2880fbff
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:54:09.702862 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:09.689859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:57421 (local address 127.17.54.254:33757)
0812 03:54:09.689957 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:09.689969 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:09.690380 (+ 411us) server_negotiation.cc:407] Connection header received
0812 03:54:09.691501 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:09.691502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:09.691522 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:09.691555 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:54:09.692854 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.693719 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.695422 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.695825 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.695910 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:09.699121 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:54:09.699123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:54:09.699123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:54:09.699133 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:54:09.701022 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:09.701583 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:09.701584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:09.701585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:09.701660 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:09.702007 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:09.702008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:09.702009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:09.702185 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:54:09.702263 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:09.702667 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 03:54:09.702763 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:54:09.703804 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:09.689812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33757 (local address 127.17.54.194:57421)
0812 03:54:09.690157 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:09.690231 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 03:54:09.691308 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:54:09.691758 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:54:09.691762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:54:09.692229 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:54:09.692703 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:09.692710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.693867 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.693868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:09.695295 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:09.695298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.695951 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.695952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:09.696178 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:09.696870 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:54:09.696882 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:54:09.699011 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:54:09.701133 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:54:09.701141 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:54:09.701144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:54:09.701464 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:54:09.701743 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:54:09.701744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:54:09.701745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:54:09.701903 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:54:09.702271 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:54:09.702278 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:54:09.702485 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 03:54:09.703303 (+ 818us) client_negotiation.cc:240] Negotiation successful
0812 03:54:09.703486 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":173,"threads_started":1}
I20260812 03:54:09.704703 30928 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33757
I20260812 03:54:09.704830 30928 heartbeater.cc:464] Registering TS with master...
I20260812 03:54:09.705060 30928 heartbeater.cc:511] Master 127.17.54.254:33757 requested a full tablet report, sending...
I20260812 03:54:09.705554 30515 ts_manager.cc:194] Registered new tserver with Master: 99daf5feda3647f0a509c11ff5a8555f (127.17.54.194:43543)
I20260812 03:54:09.706094 30515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:57421
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:54:09.730640 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:33021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33757
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:54:09.878535 30934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:54:09.878908 30934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:54:09.879009 30934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:54:09.880350 30934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:54:09.880431 30934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:54:09.880484 30934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:54:09.882210 30934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.54.212:33021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:54:09.882519 30934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:54:09.882767 30934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:54:09.889340 30942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:54:09.889595 30940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:54:09.890009 30934 server_base.cc:1034] running on GCE node
W20260812 03:54:09.890240 30944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:54:09.890547 30934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:54:09.890854 30934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:54:09.892040 30934 hybrid_clock.cc:648] HybridClock initialized: now 1786506849892029 us; error 27 us; skew 500 ppm
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:54:09.895318 30934 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:54:09.895829 30934 webserver.cc:460] Webserver started at http://127.17.54.195:45183/ using document root <none> and password file <none>
I20260812 03:54:09.896240 30934 fs_manager.cc:359] Metadata directory not provided
I20260812 03:54:09.896473 30934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:54:09.896656 30934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:54:09 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:54:09.910511 30934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "489426f6538d4cea83290e6a41dc3538"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "3701566e11c52c2ca6af80c675c14d7c"
server_key_iv: "ecfb7f9d3e0805905109dac76a2feed0"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.911208 30934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "489426f6538d4cea83290e6a41dc3538"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "3701566e11c52c2ca6af80c675c14d7c"
server_key_iv: "ecfb7f9d3e0805905109dac76a2feed0"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.913460 30934 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:54:09.929047 30951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.929316 30934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 03:54:09.929423 30934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "489426f6538d4cea83290e6a41dc3538"
format_stamp: "Formatted at 2026-08-12 03:54:09 on dist-test-slave-tm1g"
server_key: "3701566e11c52c2ca6af80c675c14d7c"
server_key_iv: "ecfb7f9d3e0805905109dac76a2feed0"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:09.929539 30934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:54:09.966717 30934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:54:09.969627 30934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:54:09.969861 30934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:54:09.970278 30934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:54:09.970908 30934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:54:09.971007 30934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.971086 30934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:54:09.971143 30934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:09.981675 30934 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:45737
I20260812 03:54:09.982025 31098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:45737 every 8 connection(s)
I20260812 03:54:09.982426 30934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:54:09.991214 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 30934
I20260812 03:54:09.991345 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:54:09.991551 17627 external_mini_cluster.cc:1428] Setting key 1d2b7c443bef06068c85aaec5feb6756
Aug 12 03:54:10 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:54:10.008634 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:09.995297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:37545 (local address 127.17.54.254:33757)
0812 03:54:09.995360 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:09.995363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:09.995748 (+ 385us) server_negotiation.cc:407] Connection header received
0812 03:54:09.996560 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:09.996561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:09.996575 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:09.996607 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:54:09.997631 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.998474 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.000094 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.000388 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.000440 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.004788 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:54:10.004790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:54:10.004790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:54:10.004800 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:54:10.006770 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:10.007412 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:10.007413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:10.007414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:10.007480 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:10.007802 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:10.007803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:10.007804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:10.007967 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:54:10.008015 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:10.008429 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 03:54:10.008531 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:54:10.009577 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:09.995291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33757 (local address 127.17.54.195:37545)
0812 03:54:09.995616 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:09.995662 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:54:09.996427 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:54:09.996821 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:54:09.996826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:54:09.997115 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:54:09.997509 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:09.997517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:09.998601 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:09.998602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:09.999968 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:09.999971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.000545 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.000546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.000743 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.001546 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:54:10.001557 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:54:10.004646 (+ 3089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:54:10.006905 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:54:10.006916 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:54:10.006919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:54:10.007299 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:54:10.007561 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:54:10.007561 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:54:10.007562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:54:10.007719 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:54:10.008070 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:54:10.008077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:54:10.008270 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 03:54:10.009082 (+ 812us) client_negotiation.cc:240] Negotiation successful
0812 03:54:10.009315 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":115,"threads_started":1}
I20260812 03:54:10.010391 31099 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33757
I20260812 03:54:10.010499 31099 heartbeater.cc:464] Registering TS with master...
I20260812 03:54:10.010707 31099 heartbeater.cc:511] Master 127.17.54.254:33757 requested a full tablet report, sending...
I20260812 03:54:10.011220 30515 ts_manager.cc:194] Registered new tserver with Master: 489426f6538d4cea83290e6a41dc3538 (127.17.54.195:45737)
I20260812 03:54:10.011725 30515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:37545
I20260812 03:54:10.015420 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:54:10.030576 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.018062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.17.54.254:33757)
0812 03:54:10.018125 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:10.018127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:10.018550 (+ 423us) server_negotiation.cc:407] Connection header received
0812 03:54:10.018662 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:10.018663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:10.018676 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:10.018709 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:54:10.019557 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.020322 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.021479 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.021740 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.021789 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.026815 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:54:10.026817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:54:10.026818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:54:10.026828 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:54:10.028784 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:10.029566 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:10.029568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:10.029568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:10.029635 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:10.029859 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:10.029860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:10.029861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:10.030020 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:54:10.030103 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:10.030370 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 03:54:10.030468 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:54:10.033429 30515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:54:10.036128 30515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:54:10.056584 31118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.041523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:38983 (local address 127.0.0.1:52602)
0812 03:54:10.042919 (+ 1396us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:10.042974 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:54:10.043046 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:54:10.050832 (+ 7786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:54:10.050833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:54:10.050835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:54:10.050895 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.050897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.053461 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.053462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.054841 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.054845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.056227 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.056228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.056336 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.056352 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 03:54:10.056423 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:54:10.056487 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"thread_start_us":108,"threads_started":1}
I20260812 03:54:10.056946 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.041551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43543 (local address 127.0.0.1:43218)
0812 03:54:10.043172 (+ 1621us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:10.043182 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:54:10.043227 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:54:10.043611 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:54:10.043614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:54:10.043617 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:54:10.043698 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.043700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.047105 (+ 3405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.047106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.052239 (+ 5133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.052244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.056640 (+ 4396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.056641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.056727 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.056741 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:54:10.056805 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:54:10.056852 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1603,"thread_start_us":125,"threads_started":1}
I20260812 03:54:10.057225 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.041854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43218 (local address 127.17.54.194:43543)
0812 03:54:10.042819 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:10.042825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:10.043287 (+ 462us) server_negotiation.cc:407] Connection header received
0812 03:54:10.043339 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:10.043342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:10.043359 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:10.043451 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:54:10.043787 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.047004 (+ 3217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.052397 (+ 5393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.056375 (+ 3978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.056478 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.056596 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:10.057085 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 03:54:10.057145 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":85,"threads_started":1}
I20260812 03:54:10.057713 31120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.042045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.17.54.193:38983)
0812 03:54:10.050541 (+ 8496us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:10.050548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:10.050565 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:54:10.050607 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:10.050609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:10.050628 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:10.050735 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:54:10.050997 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.053357 (+ 2360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.055119 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.056013 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.056094 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.057556 (+ 1462us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:10.057587 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 03:54:10.057634 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8445,"thread_start_us":58,"threads_started":1}
I20260812 03:54:10.058188 30852 tablet_service.cc:1467] Processing CreateTablet for tablet bc9b5a2910ea47fea6b663becffdcc64 (DEFAULT_TABLE table=test-table [id=0a4f7bc74cec416b80f2bf3667f3bd52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:54:10.058720 30852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9b5a2910ea47fea6b663becffdcc64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:54:10.059019 30688 tablet_service.cc:1467] Processing CreateTablet for tablet bc9b5a2910ea47fea6b663becffdcc64 (DEFAULT_TABLE table=test-table [id=0a4f7bc74cec416b80f2bf3667f3bd52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:54:10.059368 30688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9b5a2910ea47fea6b663becffdcc64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:54:10.068017 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.041777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:45737 (local address 127.0.0.1:60480)
0812 03:54:10.052036 (+ 10259us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:10.052056 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:54:10.052133 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:54:10.052820 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:54:10.052821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:54:10.052823 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:54:10.052931 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.052934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.065046 (+ 12112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.065048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.066352 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.066357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.067703 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.067705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.067793 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.067799 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:54:10.067851 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:54:10.067915 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10227,"thread_start_us":93,"threads_started":1}
I20260812 03:54:10.068593 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.042189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.17.54.195:45737)
0812 03:54:10.052365 (+ 10176us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:10.052382 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:10.052401 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:54:10.052485 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:10.052487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:10.052521 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:10.052666 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:54:10.054961 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.064907 (+ 9946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.066543 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.067456 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.067540 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.067656 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:10.068458 (+ 802us) server_negotiation.cc:299] Negotiation successful
0812 03:54:10.068509 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10086,"thread_start_us":94,"threads_started":1}
I20260812 03:54:10.069546 31013 tablet_service.cc:1467] Processing CreateTablet for tablet bc9b5a2910ea47fea6b663becffdcc64 (DEFAULT_TABLE table=test-table [id=0a4f7bc74cec416b80f2bf3667f3bd52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:54:10.069963 31013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9b5a2910ea47fea6b663becffdcc64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:54:10.074536 31123 tablet_bootstrap.cc:492] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Bootstrap starting.
I20260812 03:54:10.074671 31125 tablet_bootstrap.cc:492] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: Bootstrap starting.
I20260812 03:54:10.075817 31123 tablet_bootstrap.cc:654] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Neither blocks nor log segments found. Creating new log.
I20260812 03:54:10.075817 31125 tablet_bootstrap.cc:654] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: Neither blocks nor log segments found. Creating new log.
I20260812 03:54:10.076598 31125 log.cc:824] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: Log is configured to *not* fsync() on all Append() calls
I20260812 03:54:10.076620 31123 log.cc:824] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:54:10.076946 31124 tablet_bootstrap.cc:492] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce: Bootstrap starting.
I20260812 03:54:10.078140 31124 tablet_bootstrap.cc:654] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce: Neither blocks nor log segments found. Creating new log.
I20260812 03:54:10.078949 31124 log.cc:824] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce: Log is configured to *not* fsync() on all Append() calls
I20260812 03:54:10.080190 31125 tablet_bootstrap.cc:492] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: No bootstrap required, opened a new log
I20260812 03:54:10.080318 31125 ts_tablet_manager.cc:1397] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260812 03:54:10.083218 31123 tablet_bootstrap.cc:492] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: No bootstrap required, opened a new log
I20260812 03:54:10.083443 31123 ts_tablet_manager.cc:1397] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260812 03:54:10.083539 31125 raft_consensus.cc:348] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
I20260812 03:54:10.083874 31125 raft_consensus.cc:374] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:54:10.083921 31125 raft_consensus.cc:729] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489426f6538d4cea83290e6a41dc3538, State: Initialized, Role: FOLLOWER
I20260812 03:54:10.084081 31125 consensus_queue.cc:260] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
I20260812 03:54:10.084394 31125 ts_tablet_manager.cc:1428] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 03:54:10.084543 31099 heartbeater.cc:503] Master 127.17.54.254:33757 was elected leader, sending a full tablet report...
I20260812 03:54:10.086484 31124 tablet_bootstrap.cc:492] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce: No bootstrap required, opened a new log
I20260812 03:54:10.086587 31124 ts_tablet_manager.cc:1397] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260812 03:54:10.086889 31123 raft_consensus.cc:348] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
I20260812 03:54:10.087155 31123 raft_consensus.cc:374] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:54:10.087285 31123 raft_consensus.cc:729] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99daf5feda3647f0a509c11ff5a8555f, State: Initialized, Role: FOLLOWER
I20260812 03:54:10.087527 31123 consensus_queue.cc:260] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
I20260812 03:54:10.087937 31123 ts_tablet_manager.cc:1428] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 03:54:10.088126 30928 heartbeater.cc:503] Master 127.17.54.254:33757 was elected leader, sending a full tablet report...
I20260812 03:54:10.090240 31124 raft_consensus.cc:348] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
I20260812 03:54:10.090623 31124 raft_consensus.cc:374] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:54:10.090674 31124 raft_consensus.cc:729] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c53be68db3f49a3afa05d24368e78ce, State: Initialized, Role: FOLLOWER
I20260812 03:54:10.090843 31124 consensus_queue.cc:260] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
I20260812 03:54:10.091188 31124 ts_tablet_manager.cc:1428] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260812 03:54:10.091316 30769 heartbeater.cc:503] Master 127.17.54.254:33757 was elected leader, sending a full tablet report...
I20260812 03:54:10.126188 31132 raft_consensus.cc:482] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:54:10.126391 31132 raft_consensus.cc:504] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
W20260812 03:54:10.127921 30770 tablet.cc:2369] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:54:10.127979 31132 leader_election.cc:288] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99daf5feda3647f0a509c11ff5a8555f (127.17.54.194:43543), 489426f6538d4cea83290e6a41dc3538 (127.17.54.195:45737)
I20260812 03:54:10.136315 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.127446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:51377 (local address 127.17.54.194:43543)
0812 03:54:10.127503 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:10.127506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:10.128455 (+ 949us) server_negotiation.cc:407] Connection header received
0812 03:54:10.128590 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:10.128591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:10.128607 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:10.128637 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:54:10.129009 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.129820 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.134681 (+ 4861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.135210 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.135270 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.135280 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:10.136176 (+ 896us) server_negotiation.cc:299] Negotiation successful
0812 03:54:10.136232 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:54:10.136862 31134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.127080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43543 (local address 127.17.54.193:51377)
0812 03:54:10.128374 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:10.128387 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:54:10.128508 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:54:10.128707 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:54:10.128707 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:54:10.128718 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:54:10.128918 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.128920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.130024 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.130024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.134572 (+ 4548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.134575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.135348 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.135349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.135518 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.135523 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:54:10.136682 (+ 1159us) client_negotiation.cc:240] Negotiation successful
0812 03:54:10.136740 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1271,"thread_start_us":180,"threads_started":1}
I20260812 03:54:10.136902 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.127914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:46769 (local address 127.17.54.195:45737)
0812 03:54:10.127959 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:10.127961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:10.131617 (+ 3656us) server_negotiation.cc:407] Connection header received
0812 03:54:10.131724 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:10.131726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:10.131741 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:10.131771 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:54:10.132032 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.132812 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.134423 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.134992 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.135075 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.135086 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:10.136757 (+ 1671us) server_negotiation.cc:299] Negotiation successful
0812 03:54:10.136815 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:54:10.137252 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.127675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:45737 (local address 127.17.54.193:46769)
0812 03:54:10.131532 (+ 3857us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:10.131544 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:54:10.131649 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:54:10.131833 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:54:10.131833 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:54:10.131835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:54:10.131947 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.131949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.133016 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.133017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.134341 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:54:10.134344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.135207 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.135209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:54:10.135339 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.135345 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:54:10.137025 (+ 1680us) client_negotiation.cc:240] Negotiation successful
0812 03:54:10.137072 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3834,"thread_start_us":128,"threads_started":1}
I20260812 03:54:10.137773 31037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc9b5a2910ea47fea6b663becffdcc64" candidate_uuid: "1c53be68db3f49a3afa05d24368e78ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489426f6538d4cea83290e6a41dc3538" is_pre_election: true
I20260812 03:54:10.137946 31037 raft_consensus.cc:2454] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c53be68db3f49a3afa05d24368e78ce in term 0.
I20260812 03:54:10.138216 30632 leader_election.cc:302] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c53be68db3f49a3afa05d24368e78ce, 489426f6538d4cea83290e6a41dc3538; no voters:
I20260812 03:54:10.138723 31132 raft_consensus.cc:2781] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:54:10.138790 31132 raft_consensus.cc:482] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:54:10.138813 31132 raft_consensus.cc:3037] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:54:10.140265 31132 raft_consensus.cc:504] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
I20260812 03:54:10.140818 30874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc9b5a2910ea47fea6b663becffdcc64" candidate_uuid: "1c53be68db3f49a3afa05d24368e78ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99daf5feda3647f0a509c11ff5a8555f"
I20260812 03:54:10.140967 30874 raft_consensus.cc:3037] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:54:10.141008 31037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc9b5a2910ea47fea6b663becffdcc64" candidate_uuid: "1c53be68db3f49a3afa05d24368e78ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489426f6538d4cea83290e6a41dc3538"
I20260812 03:54:10.141085 31037 raft_consensus.cc:3037] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:54:10.142138 31037 raft_consensus.cc:2454] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c53be68db3f49a3afa05d24368e78ce in term 1.
I20260812 03:54:10.142139 30874 raft_consensus.cc:2454] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c53be68db3f49a3afa05d24368e78ce in term 1.
I20260812 03:54:10.144241 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc9b5a2910ea47fea6b663becffdcc64" candidate_uuid: "1c53be68db3f49a3afa05d24368e78ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99daf5feda3647f0a509c11ff5a8555f" is_pre_election: true
I20260812 03:54:10.144357 30875 raft_consensus.cc:2362] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1c53be68db3f49a3afa05d24368e78ce in term 1. Re-sending same reply.
I20260812 03:54:10.144691 30636 leader_election.cc:302] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c53be68db3f49a3afa05d24368e78ce, 99daf5feda3647f0a509c11ff5a8555f; no voters:
I20260812 03:54:10.145404 31132 leader_election.cc:288] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [CANDIDATE]: Term 1 election: Requested vote from peers 99daf5feda3647f0a509c11ff5a8555f (127.17.54.194:43543), 489426f6538d4cea83290e6a41dc3538 (127.17.54.195:45737)
I20260812 03:54:10.145551 31132 raft_consensus.cc:2781] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:54:10.145637 31132 raft_consensus.cc:686] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [term 1 LEADER]: Becoming Leader. State: Replica: 1c53be68db3f49a3afa05d24368e78ce, State: Running, Role: LEADER
I20260812 03:54:10.145779 31132 consensus_queue.cc:237] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } }
I20260812 03:54:10.146862 30515 catalog_manager.cc:5495] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c53be68db3f49a3afa05d24368e78ce (127.17.54.193). New cstate: current_term: 1 leader_uuid: "1c53be68db3f49a3afa05d24368e78ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c53be68db3f49a3afa05d24368e78ce" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 38983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:54:10 dist-test-slave-tm1g krb5kdc[29954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:54:10.171054 31120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.166938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52616 (local address 127.17.54.193:38983)
0812 03:54:10.166983 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:10.166985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:10.166993 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:54:10.167054 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:10.167055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:10.167080 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:10.167155 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:54:10.167349 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.168093 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.169830 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:10.170117 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:10.170189 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:10.170444 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:54:10.170592 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:54:10.170678 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:10.170931 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 03:54:10.170955 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:54:10.175395 31037 raft_consensus.cc:1261] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 1 FOLLOWER]: Refusing update from remote peer 1c53be68db3f49a3afa05d24368e78ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:54:10.175473 30875 raft_consensus.cc:1261] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 1 FOLLOWER]: Refusing update from remote peer 1c53be68db3f49a3afa05d24368e78ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:54:10.175794 31132 consensus_queue.cc:1035] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "489426f6538d4cea83290e6a41dc3538" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:54:10.175953 31132 consensus_queue.cc:1035] T bc9b5a2910ea47fea6b663becffdcc64 P 1c53be68db3f49a3afa05d24368e78ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "99daf5feda3647f0a509c11ff5a8555f" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:54:10.183616 31142 mvcc.cc:205] Tried to move back new op lower bound from 7317532058315259904 to 7317532058198274048. Current Snapshot: MvccSnapshot[applied={T|T < 7317532058315259904}]
W20260812 03:54:10.190053 30929 tablet.cc:2369] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:54:10.193998 31140 mvcc.cc:205] Tried to move back new op lower bound from 7317532058315259904 to 7317532058198274048. Current Snapshot: MvccSnapshot[applied={T|T < 7317532058315259904}]
I20260812 03:54:10.197314 30515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:54:10.197443 30515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:54:10.198230 30515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc5692ea8ce547a68726cdd6354d2c81: Sending DeleteTablet for 3 replicas of tablet bc9b5a2910ea47fea6b663becffdcc64
I20260812 03:54:10.198892 30688 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9b5a2910ea47fea6b663becffdcc64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:54:10 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:52602
I20260812 03:54:10.198885 30852 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9b5a2910ea47fea6b663becffdcc64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:54:10 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43218
I20260812 03:54:10.198885 31013 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9b5a2910ea47fea6b663becffdcc64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:54:10 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:60480
I20260812 03:54:10.199632 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 30613
I20260812 03:54:10.210541 31157 tablet_replica.cc:330] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: stopping tablet replica
I20260812 03:54:10.210765 31157 raft_consensus.cc:2229] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:54:10.210876 31157 raft_consensus.cc:2258] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:54:10.211686 31156 tablet_replica.cc:330] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: stopping tablet replica
I20260812 03:54:10.211817 31156 raft_consensus.cc:2229] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:54:10.212522 31157 ts_tablet_manager.cc:1905] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:54:10.214895 31157 ts_tablet_manager.cc:1918] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:54:10.214980 31157 log.cc:1196] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/bc9b5a2910ea47fea6b663becffdcc64
I20260812 03:54:10.215243 31157 ts_tablet_manager.cc:1939] T bc9b5a2910ea47fea6b663becffdcc64 P 99daf5feda3647f0a509c11ff5a8555f: Deleting consensus metadata
I20260812 03:54:10.215729 30501 catalog_manager.cc:4841] TS 99daf5feda3647f0a509c11ff5a8555f (127.17.54.194:43543): tablet bc9b5a2910ea47fea6b663becffdcc64 (table test-table [id=0a4f7bc74cec416b80f2bf3667f3bd52]) successfully deleted
I20260812 03:54:10.221076 31156 raft_consensus.cc:2258] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:54:10.221932 30498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:38983 (error 108)
I20260812 03:54:10.223877 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 30776
I20260812 03:54:10.226243 31156 ts_tablet_manager.cc:1905] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:54:10.238691 31156 ts_tablet_manager.cc:1918] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:54:10.238847 31156 log.cc:1196] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/bc9b5a2910ea47fea6b663becffdcc64
I20260812 03:54:10.239657 31156 ts_tablet_manager.cc:1939] T bc9b5a2910ea47fea6b663becffdcc64 P 489426f6538d4cea83290e6a41dc3538: Deleting consensus metadata
I20260812 03:54:10.240337 30498 catalog_manager.cc:4841] TS 489426f6538d4cea83290e6a41dc3538 (127.17.54.195:45737): tablet bc9b5a2910ea47fea6b663becffdcc64 (table test-table [id=0a4f7bc74cec416b80f2bf3667f3bd52]) successfully deleted
I20260812 03:54:10.245425 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:10.245229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:38983 (local address 127.0.0.1:52620)
0812 03:54:10.245319 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 03:54:10.245373 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:38983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:54:10.245716 30498 catalog_manager.cc:4584] TS 1c53be68db3f49a3afa05d24368e78ce (127.17.54.193:38983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc9b5a2910ea47fea6b663becffdcc64: Network error: Client connection negotiation failed: client connection to 127.17.54.193:38983: connect: Connection refused (error 111)
I20260812 03:54:10.247049 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 30934
I20260812 03:54:10.259932 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 30481
I20260812 03:54:10.274221 17627 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:54:10.619678 17627 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:54:10.619867 17627 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:54:10.965590 17627 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:54:10.965 UTC [29990] LOG: received smart shutdown request
2026-08-12 03:54:10.970 UTC [29990] LOG: background worker "logical replication launcher" (PID 30002) exited with exit code 1
2026-08-12 03:54:10.970 UTC [29995] LOG: shutting down
2026-08-12 03:54:11.208 UTC [29990] LOG: database system is shut down
2026-08-12T03:54:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51994 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:54:11 dist-test-slave-tm1g krb5kdc[31169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:54:11 dist-test-slave-tm1g krb5kdc[31169](info): set up 2 sockets
Aug 12 03:54:11 dist-test-slave-tm1g krb5kdc[31169](info): commencing operation
krb5kdc: starting...
W20260812 03:54:12.441226 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:54:12 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:54:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:54:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.54.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.54.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/rangeradmin_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/rangeradmin_127.17.54.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.54.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.54.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/rangerlookup_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/rangerlookup_127.17.54.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.54.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.54.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/HTTP_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/HTTP_127.17.54.212@KRBTEST.COM.keytab.
I20260812 03:54:12.637391 17627 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task75kCRI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/postgres -l logfile start
2026-08-12 03:54:15.511 UTC [31208] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:54:15.511 UTC [31208] LOG: listening on IPv4 address "127.17.54.212", port 53379
2026-08-12 03:54:15.515 UTC [31208] LOG: listening on Unix socket "/tmp/.s.PGSQL.53379"
2026-08-12 03:54:15.539 UTC [31211] LOG: database system was shut down at 2026-08-12 03:54:13 UTC
2026-08-12 03:54:15.544 UTC [31208] LOG: database system is ready to accept connections
I20260812 03:54:16.576756 17627 mini_postgres.cc:96] Postgres bound to 53379
2026-08-12 03:54:16.584 UTC [31220] FATAL: database "slave" does not exist
127.17.54.212:53379 - accepting connections
I20260812 03:54:16.584990 17627 mini_ranger.cc:161] Starting Ranger...
I20260812 03:54:16.601231 17627 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:54:17.162441 17627 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:54:17.162536 17627 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-admin
2026-08-12 03:54:17,463 [I] DB FLAVOR :POSTGRES
2026-08-12 03:54:17,464 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:54:17,464 [I] Checking connection..
2026-08-12 03:54:17,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:54:17,727 [I] Checking connection passed.
2026-08-12 03:54:17,727 [I] --------- Verifying version history table ---------
2026-08-12 03:54:17,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:54:18,009 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:54:18,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:54:18,254 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:54:18,254 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:54:18,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:54:18.527 UTC [31301] WARNING: there is no transaction in progress
2026-08-12 03:54:18,535 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:54:18,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:54:18,799 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:54:18,800 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:54:18,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:54:19,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:54:19,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tm1g.c.gcp-upstream.internal','N') ;"
2026-08-12 03:54:19,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:54:19,903 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:54:19,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:54:20,165 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:54:20,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:54:20,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:54:20,688 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:54:20,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:54:21.773 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.793 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.812 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.831 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.852 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.873 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.935 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.943 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.960 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.972 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.983 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:21.999 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:22.012 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:22.020 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:22.029 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:22.038 UTC [31427] WARNING: there is no transaction in progress
2026-08-12 03:54:22,797 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:54:22,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:54:23,060 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:54:23,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:54:23,337 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:54:23,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:54:23,588 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:54:23,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:54:23,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:54:24,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tm1g.c.gcp-upstream.internal';"
2026-08-12 03:54:24,417 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:54:24,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:54:24,690 [I] DB_PATCHES have already been applied
I20260812 03:54:24.695910 17627 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-admin:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task75kCRI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:54:24.695991 17627 mini_ranger.cc:193] Using host: 127.17.54.212
I20260812 03:54:24.699885 17627 mini_ranger.cc:227] Ranger admin URL: http://127.17.54.212:57431
Aug 12, 2026 3:54:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:54:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:54:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:54:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:54:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:54:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:54:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:54:26.196893 17627 mini_ranger.cc:160] Time spent starting Ranger: real 9.612s user 0.001s sys 0.010s
Aug 12, 2026 3:54:48 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:54:48 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 3:54:48 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:54:49 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:54:50 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:54:50.602988 17627 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.54.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.54.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/rangerkms_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.54.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/rangerkms_127.17.54.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.54.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.54.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.54.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/HTTP_127.17.54.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.54.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/HTTP_127.17.54.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:54:50 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506890, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:54:50.698293 17627 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:54:50.709532 17627 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:54:51.854616 17627 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:54:51.854696 17627 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms
I20260812 03:54:51.854712 17627 mini_ranger_kms.cc:227] Using postgres at 127.17.54.212:53379
2026-08-12 03:54:52,032 [I] DB FLAVOR :POSTGRES
2026-08-12 03:54:52,033 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:54:52,033 [I] Checking connection
2026-08-12 03:54:52,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:54:52,316 [I] connection success
2026-08-12 03:54:52,316 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:54:52,316 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:54:52,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:54:52,567 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:54:52,567 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:54:52,567 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:54:52,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.54.212:53379/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:54:52,844 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:54:53.368548 17627 mini_ranger_kms.cc:310] Created kms service
I20260812 03:54:53.573251 17627 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:54:53.619614 17627 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:54:53 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506890, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:54:54.079068 17627 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:54:54.079283 17627 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms:/tmp/dist-test-task75kCRI/build/release/bin/postgresql.jar:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task75kCRI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:54:54.079306 17627 mini_ranger_kms.cc:240] Using host: 127.17.54.212
I20260812 03:54:54.083559 17627 mini_ranger_kms.cc:276] Ranger KMS PID: 31645
I20260812 03:54:54.083648 17627 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.54.212:52575
Aug 12, 2026 3:54:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:54:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:54:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:54:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:54:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:54:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:54:56 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506896, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.54.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:54:56.526896 17627 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.829s user 0.000s sys 0.162s
I20260812 03:54:56.527019 17627 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:54:56.527079 17627 mini_ranger_kms.cc:401] 127.17.54.212:52575/kms/v1/keys
Aug 12 03:54:56 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506896, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.54.212@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
Aug 12 03:54:57 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506896, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.54.212@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
Aug 12 03:54:59 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:54:59.662575 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42617
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:42617
--ranger_config_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:52575/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:54:59.750535 31703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:54:59.750756 31703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:54:59.750779 31703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:54:59.751945 31703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:54:59.751983 31703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:54:59.751998 31703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:54:59.752010 31703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:54:59.752023 31703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:54:59.753582 31703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.54.212:52575/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:42617
--ranger_config_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:54:59.753793 31703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:54:59.753971 31703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:54:59.759775 31712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:54:59.759781 31709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:54:59.759804 31703 server_base.cc:1034] running on GCE node
W20260812 03:54:59.759781 31708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:54:59.760275 31703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:54:59.760633 31703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:54:59.761833 31703 hybrid_clock.cc:648] HybridClock initialized: now 1786506899761814 us; error 47 us; skew 500 ppm
Aug 12 03:54:59 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506899, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:54:59.764577 31703 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:54:59.765081 31703 webserver.cc:460] Webserver started at http://127.17.54.254:45879/ using document root <none> and password file <none>
I20260812 03:54:59.765370 31703 fs_manager.cc:359] Metadata directory not provided
I20260812 03:54:59.765443 31703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:54:59.765610 31703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:54:59 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506899, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:54:59.825645 31703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "2f6abc1455a84194b3a92b0b3206ca88"
format_stamp: "Formatted at 2026-08-12 03:54:59 on dist-test-slave-tm1g"
server_key: "76de1c4548ba4ce3b2361bfe520c7049"
server_key_iv: "30292edc7fb1369f561934ba3da58bc5"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:59.826125 31703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "2f6abc1455a84194b3a92b0b3206ca88"
format_stamp: "Formatted at 2026-08-12 03:54:59 on dist-test-slave-tm1g"
server_key: "76de1c4548ba4ce3b2361bfe520c7049"
server_key_iv: "30292edc7fb1369f561934ba3da58bc5"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:59.828080 31703 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 03:54:59.842346 31720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:54:59.842612 31703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 03:54:59.842765 31703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "2f6abc1455a84194b3a92b0b3206ca88"
format_stamp: "Formatted at 2026-08-12 03:54:59 on dist-test-slave-tm1g"
server_key: "76de1c4548ba4ce3b2361bfe520c7049"
server_key_iv: "30292edc7fb1369f561934ba3da58bc5"
server_key_version: "kuduclusterkey@0"
I20260812 03:54:59.842883 31703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:54:59.873178 31703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:54:59.875385 31703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:54:59.875520 31703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:54:59.880208 31703 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:42617
I20260812 03:54:59.880242 31781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:42617 every 8 connection(s)
I20260812 03:54:59.880621 31703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:54:59.881328 31782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:54:59.883919 31782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Bootstrap starting.
I20260812 03:54:59.884562 31782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Neither blocks nor log segments found. Creating new log.
I20260812 03:54:59.884882 31782 log.cc:824] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Log is configured to *not* fsync() on all Append() calls
I20260812 03:54:59.885725 31782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: No bootstrap required, opened a new log
I20260812 03:54:59.888276 31782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } }
I20260812 03:54:59.888444 31782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:54:59.888469 31782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f6abc1455a84194b3a92b0b3206ca88, State: Initialized, Role: FOLLOWER
I20260812 03:54:59.888662 31782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } }
I20260812 03:54:59.888749 31782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:54:59.888773 31782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:54:59.888829 31782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:54:59.889465 31782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } }
I20260812 03:54:59.889564 31782 leader_election.cc:302] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f6abc1455a84194b3a92b0b3206ca88; no voters:
I20260812 03:54:59.889766 31782 leader_election.cc:288] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:54:59.889938 31785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:54:59.890103 31782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:54:59.890141 31785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 1 LEADER]: Becoming Leader. State: Replica: 2f6abc1455a84194b3a92b0b3206ca88, State: Running, Role: LEADER
I20260812 03:54:59.890336 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 31703
I20260812 03:54:59.890458 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:54:59.890445 31785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } }
I20260812 03:54:59.890712 17627 external_mini_cluster.cc:1428] Setting key 5cf4366f629066c9981c31d478265a63
I20260812 03:54:59.890851 31787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f6abc1455a84194b3a92b0b3206ca88. Latest consensus state: current_term: 1 leader_uuid: "2f6abc1455a84194b3a92b0b3206ca88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } } }
I20260812 03:54:59.890946 31787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: This master's current role is: LEADER
I20260812 03:54:59.891212 31794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:54:59.891294 31786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f6abc1455a84194b3a92b0b3206ca88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } } }
I20260812 03:54:59.891449 31786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: This master's current role is: LEADER
I20260812 03:54:59.891844 31794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:54:59.892652 31782 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:54:59.896824 31794 catalog_manager.cc:1347] Generated new cluster ID: 467902fe4e084675b813f7c997a4e45f
I20260812 03:54:59.897030 31794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:54:59 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:54:59.907909 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:54:59.891467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51506 (local address 127.17.54.254:42617)
0812 03:54:59.893050 (+ 1583us) server_negotiation.cc:206] Beginning negotiation
0812 03:54:59.893060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:54:59.893094 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:54:59.894994 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:54:59.895065 (+ 71us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:54:59.895317 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:54:59.895577 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:54:59.896456 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:59.898279 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:59.899513 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:54:59.900249 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:54:59.900390 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:54:59.903412 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:54:59.903416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:54:59.903416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:54:59.903430 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:54:59.905637 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:59.906084 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:59.906088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:59.906088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:59.906194 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:54:59.906451 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:54:59.906452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:54:59.906452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:54:59.906807 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:54:59.907424 (+ 617us) server_negotiation.cc:1035] Waiting for connection context
0812 03:54:59.907517 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 03:54:59.907658 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534,"thread_start_us":117,"threads_started":1}
I20260812 03:54:59.922993 31794 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:54:59.923584 31794 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:54:59.929075 31794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Generated new TSK 0
I20260812 03:54:59.929301 31794 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:55:00.647535 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:52575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42617
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:00.754933 31831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:00.755173 31831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:00.755218 31831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:00.756569 31831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:00.756644 31831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:00.756690 31831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:55:00.758363 31831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.54.212:52575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:00.758662 31831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:00.758915 31831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:00.765374 31843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:00.765441 31837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:00.765384 31839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:00.766026 31831 server_base.cc:1034] running on GCE node
I20260812 03:55:00.766247 31831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:00.766536 31831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:00.767712 31831 hybrid_clock.cc:648] HybridClock initialized: now 1786506900767698 us; error 32 us; skew 500 ppm
Aug 12 03:55:00 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506900, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:00.770607 31831 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:55:00.771184 31831 webserver.cc:460] Webserver started at http://127.17.54.193:33229/ using document root <none> and password file <none>
I20260812 03:55:00.771517 31831 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:00.771603 31831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:00.771754 31831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:55:00 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506900, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:55:00.788938 31831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "564ac4df5a8247a7b0de289ad680f47a"
format_stamp: "Formatted at 2026-08-12 03:55:00 on dist-test-slave-tm1g"
server_key: "d0d37e52d32cf7c4b6189576a63c2e0c"
server_key_iv: "c2cd15fcb53946f4d4e026578c9f0744"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:00.789575 31831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "564ac4df5a8247a7b0de289ad680f47a"
format_stamp: "Formatted at 2026-08-12 03:55:00 on dist-test-slave-tm1g"
server_key: "d0d37e52d32cf7c4b6189576a63c2e0c"
server_key_iv: "c2cd15fcb53946f4d4e026578c9f0744"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:00.792325 31831 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:55:00.807070 31850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:00.807701 31831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:00.807822 31831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "564ac4df5a8247a7b0de289ad680f47a"
format_stamp: "Formatted at 2026-08-12 03:55:00 on dist-test-slave-tm1g"
server_key: "d0d37e52d32cf7c4b6189576a63c2e0c"
server_key_iv: "c2cd15fcb53946f4d4e026578c9f0744"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:00.807952 31831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:00.820930 31831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:00.824435 31831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:00.824658 31831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:00.824921 31831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:00.825379 31831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:00.825472 31831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:00.825549 31831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:00.825590 31831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:00.833431 31831 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:40515
I20260812 03:55:00.834007 31831 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:55:00.835922 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 31831
I20260812 03:55:00.836025 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:55:00.836225 17627 external_mini_cluster.cc:1428] Setting key faf95478f906ddee9c32bf5c8c160426
I20260812 03:55:00.846660 31982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:40515 every 8 connection(s)
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Aug 12 03:55:00 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506900, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
I20260812 03:55:00.870658 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:00.847755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42617 (local address 127.17.54.193:46943)
0812 03:55:00.851884 (+ 4129us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:00.851960 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:00.852669 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:00.853109 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:00.853115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:00.853374 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:00.853708 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:00.853715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:00.854840 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:00.854842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:00.856201 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:00.856205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:00.856816 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:00.856817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:00.857002 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:00.857727 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:00.857739 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:00.865762 (+ 8023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:00.868578 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:00.868593 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:00.868597 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:00.869021 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:00.869308 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:00.869309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:00.869309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:00.869487 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:00.869840 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:00.869847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:00.870055 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 03:55:00.870160 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:55:00.870388 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4062,"thread_start_us":105,"threads_started":1}
I20260812 03:55:00.871637 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:00.847756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:46943 (local address 127.17.54.254:42617)
0812 03:55:00.851372 (+ 3616us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:00.851375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:00.852834 (+ 1459us) server_negotiation.cc:407] Connection header received
0812 03:55:00.852871 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:00.852872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:00.852890 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:00.852958 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:00.853855 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:00.854721 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:00.856318 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:00.856646 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:00.856693 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:00.865954 (+ 9261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:00.865957 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:00.865957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:00.865967 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:00.868419 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:00.869138 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:00.869139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:00.869139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:00.869218 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:00.869581 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:00.869582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:00.869583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:00.869750 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:00.871348 (+ 1598us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:00.871399 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:55:00.871522 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3573,"thread_start_us":117,"threads_started":1}
I20260812 03:55:00.872406 31984 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42617
I20260812 03:55:00.872524 31984 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:00.872880 31984 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:00.873601 31740 ts_manager.cc:194] Registered new tserver with Master: 564ac4df5a8247a7b0de289ad680f47a (127.17.54.193:40515)
I20260812 03:55:00.874775 31740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:46943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:55:00.891120 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:52575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42617
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:01.022541 31990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:01.023077 31990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:01.023185 31990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:01.025192 31990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:01.025362 31990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:01.025456 31990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:55:01.028045 31990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.54.212:52575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:01.028618 31990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:01.029034 31990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:01.036741 31997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:01.037177 31990 server_base.cc:1034] running on GCE node
W20260812 03:55:01.036762 32001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:01.037498 31999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:01.037874 31990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:01.038257 31990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:01.039487 31990 hybrid_clock.cc:648] HybridClock initialized: now 1786506901039465 us; error 54 us; skew 500 ppm
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:01.043604 31990 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:55:01.044289 31990 webserver.cc:460] Webserver started at http://127.17.54.194:39169/ using document root <none> and password file <none>
I20260812 03:55:01.044700 31990 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:01.044853 31990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:01.045079 31990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:55:01.067287 31990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "fa561f1ff3634f19bc26d170627317ba"
format_stamp: "Formatted at 2026-08-12 03:55:01 on dist-test-slave-tm1g"
server_key: "89c796ef895a9ddb175bc8b93a331742"
server_key_iv: "38704585e1aa063b7bcb289255e123c5"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:01.067910 31990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "fa561f1ff3634f19bc26d170627317ba"
format_stamp: "Formatted at 2026-08-12 03:55:01 on dist-test-slave-tm1g"
server_key: "89c796ef895a9ddb175bc8b93a331742"
server_key_iv: "38704585e1aa063b7bcb289255e123c5"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:01.074016 31990 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:01.090607 32011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.091231 31990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:55:01.091365 31990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "fa561f1ff3634f19bc26d170627317ba"
format_stamp: "Formatted at 2026-08-12 03:55:01 on dist-test-slave-tm1g"
server_key: "89c796ef895a9ddb175bc8b93a331742"
server_key_iv: "38704585e1aa063b7bcb289255e123c5"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:01.091490 31990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:01.112267 31990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:01.115059 31990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:01.115312 31990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:01.115676 31990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:01.116144 31990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:01.116251 31990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.116329 31990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:01.116426 31990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.124430 31990 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:41827
I20260812 03:55:01.124998 31990 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:55:01.130259 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 31990
I20260812 03:55:01.130470 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:55:01.130709 17627 external_mini_cluster.cc:1428] Setting key a3edbcc5a370b7f13d71e29310193d68
I20260812 03:55:01.130707 32144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:41827 every 8 connection(s)
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:01.152556 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:01.127464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:43491 (local address 127.17.54.254:42617)
0812 03:55:01.127648 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:01.127650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:01.134662 (+ 7012us) server_negotiation.cc:407] Connection header received
0812 03:55:01.135422 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:01.135424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:01.135441 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:01.135497 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:01.142857 (+ 7360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.143644 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.145115 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.145419 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.145492 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:01.148630 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:01.148632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:01.148632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:01.148642 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:01.150648 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:01.151242 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:01.151243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:01.151244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:01.151307 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:01.151638 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:01.151639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:01.151639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:01.151798 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:01.151850 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:01.152255 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 03:55:01.152369 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260812 03:55:01.153177 32147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:01.127450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42617 (local address 127.17.54.194:43491)
0812 03:55:01.134483 (+ 7033us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:01.134551 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:01.135283 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:01.141913 (+ 6630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:01.141921 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:01.142203 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:01.142617 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:01.142625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.143799 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.143800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:01.144989 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:01.144993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.145579 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.145586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:01.145702 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:01.146397 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:01.146407 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:01.148502 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:01.150772 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:01.150782 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:01.150786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:01.151141 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:01.151387 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:01.151388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:01.151389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:01.151556 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:01.151895 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:01.151902 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:01.152090 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 03:55:01.152686 (+ 596us) client_negotiation.cc:240] Negotiation successful
0812 03:55:01.152883 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6966,"thread_start_us":98,"threads_started":1}
I20260812 03:55:01.153721 32145 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42617
I20260812 03:55:01.153821 32145 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:01.154022 32145 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:01.154452 31740 ts_manager.cc:194] Registered new tserver with Master: fa561f1ff3634f19bc26d170627317ba (127.17.54.194:41827)
I20260812 03:55:01.154945 31740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:43491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:55:01.174817 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:52575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42617
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:01.340634 32152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:01.341159 32152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:01.341269 32152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:01.343320 32152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:01.343582 32152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:01.343693 32152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:55:01.346336 32152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.54.212:52575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:01.347211 32152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:01.347615 32152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:01.355134 32162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:01.355561 32165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:01.356499 32159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:01.356629 32152 server_base.cc:1034] running on GCE node
I20260812 03:55:01.357004 32152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:01.357324 32152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:01.358655 32152 hybrid_clock.cc:648] HybridClock initialized: now 1786506901358626 us; error 43 us; skew 500 ppm
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:01.362249 32152 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:55:01.362886 32152 webserver.cc:460] Webserver started at http://127.17.54.195:39391/ using document root <none> and password file <none>
I20260812 03:55:01.363168 32152 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:01.363224 32152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:01.363353 32152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:55:01.388795 32152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "e7fb9a998ec74933a023a00e24ca39c8"
format_stamp: "Formatted at 2026-08-12 03:55:01 on dist-test-slave-tm1g"
server_key: "4d48da659d0cd5308ef968432bf2bfe3"
server_key_iv: "7ed2929cd498e4ca9e4441838e7e9b92"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:01.389469 32152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "e7fb9a998ec74933a023a00e24ca39c8"
format_stamp: "Formatted at 2026-08-12 03:55:01 on dist-test-slave-tm1g"
server_key: "4d48da659d0cd5308ef968432bf2bfe3"
server_key_iv: "7ed2929cd498e4ca9e4441838e7e9b92"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:01.402905 32152 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.002s sys 0.000s
I20260812 03:55:01.420706 32174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.420917 32152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:01.421021 32152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "e7fb9a998ec74933a023a00e24ca39c8"
format_stamp: "Formatted at 2026-08-12 03:55:01 on dist-test-slave-tm1g"
server_key: "4d48da659d0cd5308ef968432bf2bfe3"
server_key_iv: "7ed2929cd498e4ca9e4441838e7e9b92"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:01.421136 32152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:01.446425 32152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:01.448937 32152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:01.449108 32152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:01.449343 32152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:01.449704 32152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:01.449753 32152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.449791 32152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:01.449812 32152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.456748 32152 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:42333
I20260812 03:55:01.457221 32152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:55:01.461150 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 32152
I20260812 03:55:01.461266 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:55:01.461486 17627 external_mini_cluster.cc:1428] Setting key 6762f04fb726ff1aa4d3426901d895c9
I20260812 03:55:01.460893 32306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:42333 every 8 connection(s)
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:01.475966 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:01.462277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:44707 (local address 127.17.54.254:42617)
0812 03:55:01.462342 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:01.462344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:01.462802 (+ 458us) server_negotiation.cc:407] Connection header received
0812 03:55:01.463468 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:01.463469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:01.463482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:01.463550 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:01.464306 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.465127 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.466696 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.466992 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.467046 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:01.469822 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:01.469824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:01.469824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:01.469833 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:01.471581 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:01.474821 (+ 3240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:01.474823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:01.474823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:01.474891 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:01.475202 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:01.475203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:01.475204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:01.475349 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:01.475399 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:01.475773 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 03:55:01.475867 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:55:01.476804 32310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:01.461970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42617 (local address 127.17.54.195:44707)
0812 03:55:01.462669 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:01.462719 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:01.463342 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:01.463630 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:01.463634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:01.463894 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:01.464189 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:01.464194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.465256 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.465258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:01.466589 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:01.466592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.467160 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.467161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:01.467305 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:01.467963 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:01.467983 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:01.469700 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:01.474431 (+ 4731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:01.474441 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:01.474444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:01.474727 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:01.474967 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:01.474968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:01.474969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:01.475126 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:01.475432 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:01.475438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:01.475619 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 03:55:01.476363 (+ 744us) client_negotiation.cc:240] Negotiation successful
0812 03:55:01.476558 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":115,"threads_started":1}
I20260812 03:55:01.477458 32307 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42617
I20260812 03:55:01.477556 32307 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:01.477752 32307 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:01.478107 31740 ts_manager.cc:194] Registered new tserver with Master: e7fb9a998ec74933a023a00e24ca39c8 (127.17.54.195:42333)
I20260812 03:55:01.478698 31740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:44707
I20260812 03:55:01.486894 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:55:01.487782 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 31831
I20260812 03:55:01.525238 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 31990
I20260812 03:55:01.533277 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 32152
I20260812 03:55:01.543808 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 31703
I20260812 03:55:01.556216 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42617
--webserver_interface=127.17.54.254
--webserver_port=45879
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:42617
--ranger_config_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:52575/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.54.212:52575/kms with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:01.647087 32313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:01.647380 32313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:01.647432 32313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:01.648725 32313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:01.648809 32313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:01.648835 32313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:01.648866 32313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:01.648895 32313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:01.650754 32313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.54.212:52575/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:42617
--ranger_config_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=45879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:01.651067 32313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:01.651319 32313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:01.658398 32319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:01.658398 32318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:01.658408 32321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:01.659035 32313 server_base.cc:1034] running on GCE node
I20260812 03:55:01.659323 32313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:01.659660 32313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:01.660880 32313 hybrid_clock.cc:648] HybridClock initialized: now 1786506901660848 us; error 46 us; skew 500 ppm
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:01.664222 32313 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:55:01.664978 32313 webserver.cc:460] Webserver started at http://127.17.54.254:45879/ using document root <none> and password file <none>
I20260812 03:55:01.665254 32313 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:01.665359 32313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:55:01.716361 32313 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 03:55:01.717428 32330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.717767 32313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:01.717882 32313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "2f6abc1455a84194b3a92b0b3206ca88"
format_stamp: "Formatted at 2026-08-12 03:54:59 on dist-test-slave-tm1g"
server_key: "76de1c4548ba4ce3b2361bfe520c7049"
server_key_iv: "30292edc7fb1369f561934ba3da58bc5"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:01.718427 32313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:01.749025 32313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:01.751391 32313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:01.751534 32313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:01.756958 32313 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:42617
I20260812 03:55:01.756996 32388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:42617 every 8 connection(s)
I20260812 03:55:01.757411 32313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:55:01.758996 32389 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:55:01.759754 32389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Bootstrap starting.
I20260812 03:55:01.764525 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 32313
I20260812 03:55:01.764945 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:52575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:40515
--local_ip_for_outbound_sockets=127.17.54.193
--tserver_master_addrs=127.17.54.254:42617
--webserver_port=33229
--webserver_interface=127.17.54.193
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.54.212:52575/kms with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:01.769059 32389 log.cc:824] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:01.771117 32389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:01.771445 32389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Bootstrap complete.
I20260812 03:55:01.773991 32389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } }
I20260812 03:55:01.774255 32389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f6abc1455a84194b3a92b0b3206ca88, State: Initialized, Role: FOLLOWER
I20260812 03:55:01.774495 32389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } }
I20260812 03:55:01.774586 32389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:55:01.774655 32389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:55:01.774704 32389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:55:01.775849 32389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } }
I20260812 03:55:01.775960 32389 leader_election.cc:302] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f6abc1455a84194b3a92b0b3206ca88; no voters:
I20260812 03:55:01.776211 32389 leader_election.cc:288] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:55:01.776254 32395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:55:01.776372 32395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [term 2 LEADER]: Becoming Leader. State: Replica: 2f6abc1455a84194b3a92b0b3206ca88, State: Running, Role: LEADER
I20260812 03:55:01.776520 32389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:01.776572 32395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } }
I20260812 03:55:01.777022 32396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2f6abc1455a84194b3a92b0b3206ca88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } } }
I20260812 03:55:01.777125 32396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:01.777040 32397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f6abc1455a84194b3a92b0b3206ca88. Latest consensus state: current_term: 2 leader_uuid: "2f6abc1455a84194b3a92b0b3206ca88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f6abc1455a84194b3a92b0b3206ca88" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42617 } } }
I20260812 03:55:01.777197 32397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:01.777410 32403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:55:01.778085 32403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:55:01.778257 32403 catalog_manager.cc:1259] Loaded cluster ID: 467902fe4e084675b813f7c997a4e45f
I20260812 03:55:01.778373 32403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:55:01.778605 32389 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:55:01.779757 32403 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:55:01.779954 32403 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Loaded TSK: 0
I20260812 03:55:01.780180 32403 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:55:01.880923 32391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:01.881189 32391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:01.881232 32391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:01.882658 32391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:01.882727 32391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:01.882776 32391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:55:01.884407 32391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.54.212:52575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:40515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=33229
--tserver_master_addrs=127.17.54.254:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:01.884649 32391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:01.884918 32391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:01.891870 32430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:01.891858 32431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:01.892259 32435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:01.892503 32391 server_base.cc:1034] running on GCE node
I20260812 03:55:01.892735 32391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:01.893040 32391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:01.894213 32391 hybrid_clock.cc:648] HybridClock initialized: now 1786506901894180 us; error 45 us; skew 500 ppm
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:01.897665 32391 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:55:01.898248 32391 webserver.cc:460] Webserver started at http://127.17.54.193:33229/ using document root <none> and password file <none>
I20260812 03:55:01.898574 32391 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:01.898660 32391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:55:01.919286 32391 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:55:01.920526 32444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.920904 32391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:01.920998 32391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "564ac4df5a8247a7b0de289ad680f47a"
format_stamp: "Formatted at 2026-08-12 03:55:00 on dist-test-slave-tm1g"
server_key: "d0d37e52d32cf7c4b6189576a63c2e0c"
server_key_iv: "c2cd15fcb53946f4d4e026578c9f0744"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:01.921514 32391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:01.947618 32391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:01.950034 32391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:01.950274 32391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:01.950620 32391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:01.951059 32391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:01.951114 32391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.951159 32391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:01.951180 32391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:01.958892 32391 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:40515
I20260812 03:55:01.958950 32575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:40515 every 8 connection(s)
I20260812 03:55:01.959352 32391 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:55:01.967427 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 32391
I20260812 03:55:01.967746 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:52575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:41827
--local_ip_for_outbound_sockets=127.17.54.194
--tserver_master_addrs=127.17.54.254:42617
--webserver_port=39169
--webserver_interface=127.17.54.194
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.54.212:52575/kms with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:55:01 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:01.972738 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:01.960419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:51975 (local address 127.17.54.254:42617)
0812 03:55:01.960687 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:01.960697 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:01.960743 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:55:01.961572 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:01.961579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:01.961645 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:01.961809 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:01.962561 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.964016 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.965106 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.965770 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.965897 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:01.968417 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:01.968421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:01.968422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:01.968435 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:01.970544 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:01.971082 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:01.971086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:01.971086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:01.971203 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:01.971528 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:01.971529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:01.971530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:01.971848 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:01.971953 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:01.972372 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 03:55:01.972519 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":122,"threads_started":1}
I20260812 03:55:01.972754 32578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:01.960341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42617 (local address 127.17.54.193:51975)
0812 03:55:01.960596 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:01.960647 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:01.961283 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:01.961867 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:01.961871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:01.962114 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:01.962421 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:01.962428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.964137 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.964138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:01.965006 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:01.965009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:01.965886 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:01.965887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:01.965990 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:01.966688 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:01.966698 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:01.968293 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:01.970639 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:01.970647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:01.970649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:01.970984 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:01.971283 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:01.971284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:01.971285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:01.971442 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:01.971953 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:01.971960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:01.972138 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 03:55:01.972279 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 03:55:01.972474 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":101,"threads_started":1}
I20260812 03:55:01.973644 32576 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42617
I20260812 03:55:01.973769 32576 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:01.973989 32576 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
W20260812 03:55:01.975098 32576 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:42617 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:55:01.975183 32576 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:01.975260 32576 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:01.975646 32576 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:01.975739 32576 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
W20260812 03:55:01.976133 32576 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:55:02.082275 32581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:02.082616 32581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:02.082653 32581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:02.084504 32581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:02.084580 32581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:02.084625 32581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:55:02.087162 32581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.54.212:52575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:41827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=39169
--tserver_master_addrs=127.17.54.254:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:02.087477 32581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:02.087750 32581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:02.095305 32586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:02.095364 32588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:02.095559 32590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:02.096475 32581 server_base.cc:1034] running on GCE node
I20260812 03:55:02.096835 32581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:02.097162 32581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:02.098378 32581 hybrid_clock.cc:648] HybridClock initialized: now 1786506902098348 us; error 44 us; skew 500 ppm
Aug 12 03:55:02 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506902, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:02.101416 32581 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:55:02.101934 32581 webserver.cc:460] Webserver started at http://127.17.54.194:39169/ using document root <none> and password file <none>
I20260812 03:55:02.102242 32581 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:02.102357 32581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:55:02 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506902, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:55:02.118148 32581 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:55:02.119066 32602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:02.119285 32581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:02.119381 32581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "fa561f1ff3634f19bc26d170627317ba"
format_stamp: "Formatted at 2026-08-12 03:55:01 on dist-test-slave-tm1g"
server_key: "89c796ef895a9ddb175bc8b93a331742"
server_key_iv: "38704585e1aa063b7bcb289255e123c5"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:02.119897 32581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:02.135646 32581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:02.138171 32581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:02.138410 32581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:02.138702 32581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:02.139111 32581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:02.139168 32581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:02.139210 32581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:02.139235 32581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:02.147064 32581 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:41827
I20260812 03:55:02.147161 32734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:41827 every 8 connection(s)
I20260812 03:55:02.147500 32581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:55:02 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506902, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:02.155462 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 32581
I20260812 03:55:02.155761 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.54.212:52575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:42333
--local_ip_for_outbound_sockets=127.17.54.195
--tserver_master_addrs=127.17.54.254:42617
--webserver_port=39391
--webserver_interface=127.17.54.195
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.54.212:52575/kms with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:02.160883 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:02.148356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42617 (local address 127.17.54.194:44227)
0812 03:55:02.148542 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:02.148585 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:02.149205 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:02.149454 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:02.149458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:02.149656 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:02.149859 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:02.149864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.150668 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.150669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:02.151713 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:02.151717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.152161 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.152162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:02.152296 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:02.152891 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:02.152901 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:02.154572 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:02.158992 (+ 4420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:02.159003 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:02.159006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:02.159277 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:02.159547 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:02.159548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:02.159549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:02.159719 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:02.160066 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:02.160074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:02.160269 (+ 195us) client_negotiation.cc:769] Sending connection context
0812 03:55:02.160383 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:55:02.160576 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260812 03:55:02.161273 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:02.148417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:44227 (local address 127.17.54.254:42617)
0812 03:55:02.148620 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:02.148622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:02.148663 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:55:02.149324 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:02.149325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:02.149339 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:02.149386 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:02.149972 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.150574 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.151820 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.152027 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.152132 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:02.154700 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:02.154702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:02.154702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:02.154711 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:02.158885 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:02.159401 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:02.159402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:02.159402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:02.159466 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:02.159799 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:02.159800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:02.159801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:02.159974 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:02.160037 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:02.161076 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0812 03:55:02.161174 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260812 03:55:02.161877 32735 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42617
I20260812 03:55:02.162005 32735 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:02.162241 32735 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
W20260812 03:55:02.163019 32735 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:42617 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:55:02.163103 32735 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:02.163180 32735 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:02.163532 32735 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:02.163620 32735 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
W20260812 03:55:02.163949 32735 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:55:02.269405 32738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:02.269692 32738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:02.269731 32738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:02.271593 32738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:02.271665 32738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:02.271708 32738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:55:02.273932 32738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.54.212:52575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:42333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=39391
--tserver_master_addrs=127.17.54.254:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:02.274206 32738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:02.274510 32738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:02.281895 32747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:02.281913 32744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:02.282205 32743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:02.282485 32738 server_base.cc:1034] running on GCE node
I20260812 03:55:02.282708 32738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:02.282999 32738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:02.284183 32738 hybrid_clock.cc:648] HybridClock initialized: now 1786506902284153 us; error 47 us; skew 500 ppm
Aug 12 03:55:02 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506902, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:02.287583 32738 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:55:02.288156 32738 webserver.cc:460] Webserver started at http://127.17.54.195:39391/ using document root <none> and password file <none>
I20260812 03:55:02.288434 32738 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:02.288533 32738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:55:02 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506902, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:55:02.318811 32738 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.001s sys 0.000s
I20260812 03:55:02.321072 32758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:02.321259 32738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:55:02.321343 32738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "e7fb9a998ec74933a023a00e24ca39c8"
format_stamp: "Formatted at 2026-08-12 03:55:01 on dist-test-slave-tm1g"
server_key: "4d48da659d0cd5308ef968432bf2bfe3"
server_key_iv: "7ed2929cd498e4ca9e4441838e7e9b92"
server_key_version: "kuduclusterkey@0"
I20260812 03:55:02.321857 32738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:02.333139 32738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:02.335413 32738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:02.335597 32738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:02.335879 32738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:02.336279 32738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:02.336336 32738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:02.336388 32738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:02.336421 32738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:02.343849 32738 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:42333
I20260812 03:55:02.344347 32738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:55:02.345933 414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:42333 every 8 connection(s)
Aug 12 03:55:02 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506902, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:02.354288 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 32738
I20260812 03:55:02.355870 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:02.346421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:58821 (local address 127.17.54.254:42617)
0812 03:55:02.346475 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:02.346477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:02.346722 (+ 245us) server_negotiation.cc:407] Connection header received
0812 03:55:02.347272 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:02.347273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:02.347286 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:02.347333 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:02.347922 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.348494 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.349532 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.349749 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.349829 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:02.352341 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:02.352342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:02.352343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:02.352351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:02.353946 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:02.354600 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:02.354602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:02.354602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:02.354665 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:02.355195 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:02.355196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:02.355196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:02.355331 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:02.355371 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:02.355682 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 03:55:02.355776 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:55:02.356078 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:02.346246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:42617 (local address 127.17.54.195:58821)
0812 03:55:02.346590 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:02.346629 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:02.347154 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:02.347418 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:02.347422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:02.347612 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:02.347813 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:02.347817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.348599 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.348601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:02.349427 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:02.349429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.349874 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.349875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:02.349997 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:02.350672 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:02.350681 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:02.352232 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:02.354061 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:02.354103 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:02.354107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:02.354498 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:02.354936 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:02.354937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:02.354938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:02.355112 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:02.355402 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:02.355407 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:02.355535 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 03:55:02.355645 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 03:55:02.355809 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":255,"threads_started":1}
I20260812 03:55:02.356679 415 heartbeater.cc:347] Connected to a master server at 127.17.54.254:42617
I20260812 03:55:02.356791 415 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:02.357010 415 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
W20260812 03:55:02.357595 415 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:42617 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:55:02.357684 415 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:02.357752 415 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:02.358052 415 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:02.358126 415 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
W20260812 03:55:02.358412 415 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:55:02.363574 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:02.354771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.17.54.254:42617)
0812 03:55:02.354891 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:02.354893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:02.355032 (+ 139us) server_negotiation.cc:407] Connection header received
0812 03:55:02.355088 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:02.355089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:02.355102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:02.355186 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:02.356617 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.357338 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.358504 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:02.358729 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:02.358783 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:02.360133 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:02.360134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:02.360135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:02.360141 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:02.361732 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:02.362723 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:02.362724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:02.362724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:02.362785 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:02.363012 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:02.363013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:02.363013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:02.363130 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:02.363330 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:02.363368 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 03:55:02.363448 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":65,"threads_started":1}
Aug 12 03:55:02 dist-test-slave-tm1g krb5kdc[31169](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506902, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:02.976338 32576 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:02.976523 32576 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:02.977097 32349 ts_manager.cc:194] Registered new tserver with Master: 564ac4df5a8247a7b0de289ad680f47a (127.17.54.193:40515)
I20260812 03:55:02.978168 32349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:51975
I20260812 03:55:03.164170 32735 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:03.164347 32735 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:03.164698 32349 ts_manager.cc:194] Registered new tserver with Master: fa561f1ff3634f19bc26d170627317ba (127.17.54.194:41827)
I20260812 03:55:03.165232 32349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:44227
Aug 12 03:55:03 dist-test-slave-tm1g krb5kdc[31169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506902, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for HTTP/127.17.54.212@KRBTEST.COM
I20260812 03:55:03.358578 415 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:03.358744 415 heartbeater.cc:511] Master 127.17.54.254:42617 requested a full tablet report, sending...
I20260812 03:55:03.359041 32349 ts_manager.cc:194] Registered new tserver with Master: e7fb9a998ec74933a023a00e24ca39c8 (127.17.54.195:42333)
I20260812 03:55:03.359586 32349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:58821
I20260812 03:55:03.367803 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:55:03.378263 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.369227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51524 (local address 127.17.54.254:42617)
0812 03:55:03.369978 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:03.369980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:03.369990 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:55:03.370022 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:03.370023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:03.370034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:03.370135 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:03.370828 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.371649 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.372640 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.372949 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.373027 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.374576 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:03.374577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:03.374578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:03.374585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:03.376534 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:03.376980 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:03.376982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:03.376982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:03.377060 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:03.377290 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:03.377291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:03.377291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:03.377432 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:03.377521 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:03.378016 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 03:55:03.378120 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":83,"threads_started":1}
I20260812 03:55:03.381924 32349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:55:03.384859 32349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:55:03.403422 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.393264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:41827 (local address 127.0.0.1:48222)
0812 03:55:03.398376 (+ 5112us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:03.398389 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:03.398457 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:03.398909 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:03.398913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:03.398916 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:03.399001 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.399003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.400293 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.400294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.401971 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.401975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.403089 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.403090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.403173 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.403187 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:55:03.403259 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:55:03.403317 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5091,"thread_start_us":67,"threads_started":1}
I20260812 03:55:03.403767 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.394187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.17.54.194:41827)
0812 03:55:03.394627 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:03.394633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:03.398539 (+ 3906us) server_negotiation.cc:407] Connection header received
0812 03:55:03.398585 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:03.398588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:03.398692 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:03.398781 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:03.399100 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.400187 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.402074 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.402906 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.402989 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.403051 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:03.403644 (+ 593us) server_negotiation.cc:299] Negotiation successful
0812 03:55:03.403692 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":59,"threads_started":1}
I20260812 03:55:03.404493 32654 tablet_service.cc:1467] Processing CreateTablet for tablet 4b733e047fc540129de6ca37cbfba40b (DEFAULT_TABLE table=test-table [id=c726d31662c241e7aaf6e24a09058219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:03.404889 32654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b733e047fc540129de6ca37cbfba40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:03.405910 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.392987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:40515 (local address 127.0.0.1:43850)
0812 03:55:03.394959 (+ 1972us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:03.394967 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:03.395005 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:03.400616 (+ 5611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:03.400617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:03.400619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:03.400664 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.400666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.401921 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.401922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.403045 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.403048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.405663 (+ 2615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.405664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.405742 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.405747 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:55:03.405795 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 03:55:03.405834 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1955,"thread_start_us":76,"threads_started":1}
I20260812 03:55:03.407490 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.392681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:42333 (local address 127.0.0.1:56788)
0812 03:55:03.394716 (+ 2035us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:03.394761 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:03.394839 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:03.403456 (+ 8617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:03.403457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:03.403460 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:03.403524 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.403525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.404686 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.404687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.406160 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.406164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.407241 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.407243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.407320 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.407324 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:55:03.407373 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:55:03.407414 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2003,"thread_start_us":115,"threads_started":1}
I20260812 03:55:03.407810 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.392776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.17.54.195:42333)
0812 03:55:03.403225 (+ 10449us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:03.403231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:03.403242 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:55:03.403276 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:03.403278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:03.403296 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:03.403372 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:03.403601 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.404577 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.406265 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.407082 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.407156 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.407206 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:03.407686 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 03:55:03.407735 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10404,"thread_start_us":63,"threads_started":1}
I20260812 03:55:03.408278 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.393090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43850 (local address 127.17.54.193:40515)
0812 03:55:03.400355 (+ 7265us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:03.400361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:03.400372 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:55:03.400406 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:03.400408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:03.400425 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:03.400510 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:03.400756 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.401820 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.404690 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.405485 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.405569 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.405626 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:03.408140 (+ 2514us) server_negotiation.cc:299] Negotiation successful
0812 03:55:03.408192 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7224,"thread_start_us":51,"threads_started":1}
I20260812 03:55:03.408568 334 tablet_service.cc:1467] Processing CreateTablet for tablet 4b733e047fc540129de6ca37cbfba40b (DEFAULT_TABLE table=test-table [id=c726d31662c241e7aaf6e24a09058219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:03.408977 334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b733e047fc540129de6ca37cbfba40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:03.408891 32501 tablet_service.cc:1467] Processing CreateTablet for tablet 4b733e047fc540129de6ca37cbfba40b (DEFAULT_TABLE table=test-table [id=c726d31662c241e7aaf6e24a09058219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:03.409226 32501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b733e047fc540129de6ca37cbfba40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:03.414494 456 tablet_bootstrap.cc:492] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: Bootstrap starting.
I20260812 03:55:03.415205 456 tablet_bootstrap.cc:654] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:03.415531 456 log.cc:824] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:03.418357 457 tablet_bootstrap.cc:492] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a: Bootstrap starting.
I20260812 03:55:03.419070 457 tablet_bootstrap.cc:654] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:03.419399 457 log.cc:824] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:03.419790 455 tablet_bootstrap.cc:492] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: Bootstrap starting.
I20260812 03:55:03.420447 455 tablet_bootstrap.cc:654] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:03.420756 455 log.cc:824] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:03.421627 456 tablet_bootstrap.cc:492] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: No bootstrap required, opened a new log
I20260812 03:55:03.421713 456 ts_tablet_manager.cc:1397] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260812 03:55:03.425040 456 raft_consensus.cc:348] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.425242 456 raft_consensus.cc:374] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:03.425310 456 raft_consensus.cc:729] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7fb9a998ec74933a023a00e24ca39c8, State: Initialized, Role: FOLLOWER
I20260812 03:55:03.425441 456 consensus_queue.cc:260] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.425737 456 ts_tablet_manager.cc:1428] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 03:55:03.425894 415 heartbeater.cc:503] Master 127.17.54.254:42617 was elected leader, sending a full tablet report...
I20260812 03:55:03.426548 457 tablet_bootstrap.cc:492] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a: No bootstrap required, opened a new log
I20260812 03:55:03.426678 457 ts_tablet_manager.cc:1397] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260812 03:55:03.428485 455 tablet_bootstrap.cc:492] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: No bootstrap required, opened a new log
I20260812 03:55:03.428570 455 ts_tablet_manager.cc:1397] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260812 03:55:03.429558 457 raft_consensus.cc:348] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.429793 457 raft_consensus.cc:374] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:03.429886 457 raft_consensus.cc:729] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564ac4df5a8247a7b0de289ad680f47a, State: Initialized, Role: FOLLOWER
I20260812 03:55:03.430068 457 consensus_queue.cc:260] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.430441 457 ts_tablet_manager.cc:1428] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 03:55:03.430601 32576 heartbeater.cc:503] Master 127.17.54.254:42617 was elected leader, sending a full tablet report...
I20260812 03:55:03.431212 455 raft_consensus.cc:348] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.431376 455 raft_consensus.cc:374] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:03.431406 455 raft_consensus.cc:729] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa561f1ff3634f19bc26d170627317ba, State: Initialized, Role: FOLLOWER
I20260812 03:55:03.431556 455 consensus_queue.cc:260] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.431795 455 ts_tablet_manager.cc:1428] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:55:03.431830 32735 heartbeater.cc:503] Master 127.17.54.254:42617 was elected leader, sending a full tablet report...
W20260812 03:55:03.460866 32577 tablet.cc:2369] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:55:03.465951 461 raft_consensus.cc:482] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:55:03.466130 461 raft_consensus.cc:504] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.466621 461 leader_election.cc:288] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa561f1ff3634f19bc26d170627317ba (127.17.54.194:41827), 564ac4df5a8247a7b0de289ad680f47a (127.17.54.193:40515)
I20260812 03:55:03.470616 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.466838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:40515 (local address 127.17.54.195:47475)
0812 03:55:03.467166 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:03.467178 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:03.467271 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:03.467418 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:03.467419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:03.467421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:03.467527 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.467530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.468390 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.468391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.469455 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.469458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.470330 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.470331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.470405 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.470410 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:55:03.470467 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:55:03.470510 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":4,"thread_start_us":134,"threads_started":1}
I20260812 03:55:03.470628 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.467044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:47475 (local address 127.17.54.193:40515)
0812 03:55:03.467119 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:03.467122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:03.467258 (+ 136us) server_negotiation.cc:407] Connection header received
0812 03:55:03.467341 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:03.467342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:03.467352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:03.467397 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:03.467657 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.468298 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.469612 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.470203 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.470272 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.470280 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:03.470494 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 03:55:03.470535 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:55:03.470902 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.466989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:39873 (local address 127.17.54.194:41827)
0812 03:55:03.467097 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:03.467099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:03.467270 (+ 171us) server_negotiation.cc:407] Connection header received
0812 03:55:03.467406 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:03.467408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:03.467420 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:03.467477 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:03.467760 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.468530 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.469970 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.470545 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.470614 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.470621 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:03.470801 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 03:55:03.470840 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 03:55:03.470920 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.466838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:41827 (local address 127.17.54.195:39873)
0812 03:55:03.467184 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:03.467196 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:03.467298 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:03.467483 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:03.467484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:03.467486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:03.467614 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.467616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.468634 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.468635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.469834 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:03.469837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.470656 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.470657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:03.470737 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.470741 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:55:03.470796 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:55:03.470835 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":18,"thread_start_us":125,"threads_started":1}
I20260812 03:55:03.471583 32681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b733e047fc540129de6ca37cbfba40b" candidate_uuid: "e7fb9a998ec74933a023a00e24ca39c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa561f1ff3634f19bc26d170627317ba" is_pre_election: true
I20260812 03:55:03.471657 32521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b733e047fc540129de6ca37cbfba40b" candidate_uuid: "e7fb9a998ec74933a023a00e24ca39c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "564ac4df5a8247a7b0de289ad680f47a" is_pre_election: true
I20260812 03:55:03.471789 32521 raft_consensus.cc:2454] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7fb9a998ec74933a023a00e24ca39c8 in term 0.
I20260812 03:55:03.471789 32681 raft_consensus.cc:2454] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7fb9a998ec74933a023a00e24ca39c8 in term 0.
I20260812 03:55:03.472016 32760 leader_election.cc:302] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7fb9a998ec74933a023a00e24ca39c8, fa561f1ff3634f19bc26d170627317ba; no voters:
I20260812 03:55:03.472150 461 raft_consensus.cc:2781] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:55:03.472229 461 raft_consensus.cc:482] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:55:03.472252 461 raft_consensus.cc:3037] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:03.473166 461 raft_consensus.cc:504] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.473313 461 leader_election.cc:288] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [CANDIDATE]: Term 1 election: Requested vote from peers fa561f1ff3634f19bc26d170627317ba (127.17.54.194:41827), 564ac4df5a8247a7b0de289ad680f47a (127.17.54.193:40515)
I20260812 03:55:03.473558 32681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b733e047fc540129de6ca37cbfba40b" candidate_uuid: "e7fb9a998ec74933a023a00e24ca39c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa561f1ff3634f19bc26d170627317ba"
I20260812 03:55:03.473578 32521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b733e047fc540129de6ca37cbfba40b" candidate_uuid: "e7fb9a998ec74933a023a00e24ca39c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "564ac4df5a8247a7b0de289ad680f47a"
I20260812 03:55:03.473670 32521 raft_consensus.cc:3037] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:03.473672 32681 raft_consensus.cc:3037] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:03.474521 32681 raft_consensus.cc:2454] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7fb9a998ec74933a023a00e24ca39c8 in term 1.
I20260812 03:55:03.474648 32521 raft_consensus.cc:2454] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7fb9a998ec74933a023a00e24ca39c8 in term 1.
I20260812 03:55:03.474694 32760 leader_election.cc:302] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7fb9a998ec74933a023a00e24ca39c8, fa561f1ff3634f19bc26d170627317ba; no voters:
I20260812 03:55:03.474843 461 raft_consensus.cc:2781] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:03.475029 461 raft_consensus.cc:686] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 1 LEADER]: Becoming Leader. State: Replica: e7fb9a998ec74933a023a00e24ca39c8, State: Running, Role: LEADER
I20260812 03:55:03.475131 461 consensus_queue.cc:237] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } }
I20260812 03:55:03.476106 32348 catalog_manager.cc:5495] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7fb9a998ec74933a023a00e24ca39c8 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "e7fb9a998ec74933a023a00e24ca39c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7fb9a998ec74933a023a00e24ca39c8" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 42333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 } health_report { overall_health: UNKNOWN } } }
I20260812 03:55:03.493604 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.491081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.17.54.195:42333)
0812 03:55:03.491145 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:03.491147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:03.491154 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:55:03.491201 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:03.491202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:03.491219 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:03.491300 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:55:03.491437 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.491935 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.492798 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:03.493005 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:03.493082 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:03.493226 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:55:03.493347 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:55:03.493461 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:03.493487 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 03:55:03.493509 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:55:03.495776 32521 raft_consensus.cc:1261] T 4b733e047fc540129de6ca37cbfba40b P 564ac4df5a8247a7b0de289ad680f47a [term 1 FOLLOWER]: Refusing update from remote peer e7fb9a998ec74933a023a00e24ca39c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:03.495935 32681 raft_consensus.cc:1261] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 1 FOLLOWER]: Refusing update from remote peer e7fb9a998ec74933a023a00e24ca39c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:03.496124 461 consensus_queue.cc:1035] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "564ac4df5a8247a7b0de289ad680f47a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 40515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:03.496260 467 consensus_queue.cc:1035] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa561f1ff3634f19bc26d170627317ba" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:03.500519 472 mvcc.cc:205] Tried to move back new op lower bound from 7317532276715909120 to 7317532276634759168. Current Snapshot: MvccSnapshot[applied={T|T < 7317532276715909120}]
I20260812 03:55:03.501221 470 mvcc.cc:205] Tried to move back new op lower bound from 7317532276715909120 to 7317532276634759168. Current Snapshot: MvccSnapshot[applied={T|T < 7317532276715909120}]
I20260812 03:55:03.506903 473 mvcc.cc:205] Tried to move back new op lower bound from 7317532276715909120 to 7317532276634759168. Current Snapshot: MvccSnapshot[applied={T|T < 7317532276715909120}]
I20260812 03:55:03.509522 32348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:03.509647 32348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:03.510591 32348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f6abc1455a84194b3a92b0b3206ca88: Sending DeleteTablet for 3 replicas of tablet 4b733e047fc540129de6ca37cbfba40b
I20260812 03:55:03.511092 334 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b733e047fc540129de6ca37cbfba40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:03 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:56788
I20260812 03:55:03.511117 32501 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b733e047fc540129de6ca37cbfba40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:03 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43850
I20260812 03:55:03.511507 32654 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b733e047fc540129de6ca37cbfba40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:03 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:48222
I20260812 03:55:03.511855 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 32391
I20260812 03:55:03.522461 485 tablet_replica.cc:330] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: stopping tablet replica
I20260812 03:55:03.522706 485 raft_consensus.cc:2229] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:03.523039 485 raft_consensus.cc:2258] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:03.523698 485 ts_tablet_manager.cc:1905] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:03.525913 485 ts_tablet_manager.cc:1918] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:03.526095 485 log.cc:1196] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/4b733e047fc540129de6ca37cbfba40b
I20260812 03:55:03.526511 485 ts_tablet_manager.cc:1939] T 4b733e047fc540129de6ca37cbfba40b P fa561f1ff3634f19bc26d170627317ba: Deleting consensus metadata
I20260812 03:55:03.527194 32333 catalog_manager.cc:4841] TS fa561f1ff3634f19bc26d170627317ba (127.17.54.194:41827): tablet 4b733e047fc540129de6ca37cbfba40b (table test-table [id=c726d31662c241e7aaf6e24a09058219]) successfully deleted
W20260812 03:55:03.530655 32335 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:40515 (error 108)
I20260812 03:55:03.531369 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 32581
I20260812 03:55:03.534523 482 tablet_replica.cc:330] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: stopping tablet replica
I20260812 03:55:03.534695 482 raft_consensus.cc:2229] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:55:03.534885 482 raft_consensus.cc:2258] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:03.538051 482 ts_tablet_manager.cc:1905] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:03.539222 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:03.539082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:40515 (local address 127.0.0.1:43860)
0812 03:55:03.539142 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:03.539180 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:40515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:55:03.539434 32335 catalog_manager.cc:4584] TS 564ac4df5a8247a7b0de289ad680f47a (127.17.54.193:40515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b733e047fc540129de6ca37cbfba40b: Network error: Client connection negotiation failed: client connection to 127.17.54.193:40515: connect: Connection refused (error 111)
I20260812 03:55:03.542706 482 ts_tablet_manager.cc:1918] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:03.542855 482 log.cc:1196] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/4b733e047fc540129de6ca37cbfba40b
I20260812 03:55:03.543238 482 ts_tablet_manager.cc:1939] T 4b733e047fc540129de6ca37cbfba40b P e7fb9a998ec74933a023a00e24ca39c8: Deleting consensus metadata
I20260812 03:55:03.543768 32334 catalog_manager.cc:4841] TS e7fb9a998ec74933a023a00e24ca39c8 (127.17.54.195:42333): tablet 4b733e047fc540129de6ca37cbfba40b (table test-table [id=c726d31662c241e7aaf6e24a09058219]) successfully deleted
I20260812 03:55:03.548091 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 32738
I20260812 03:55:03.557301 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 32313
I20260812 03:55:03.568920 17627 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:55:03.914475 17627 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:55:03.914575 17627 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:55:04.260052 17627 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:55:04.260 UTC [31208] LOG: received smart shutdown request
2026-08-12 03:55:04.266 UTC [31208] LOG: background worker "logical replication launcher" (PID 31218) exited with exit code 1
2026-08-12 03:55:04.266 UTC [31212] LOG: shutting down
2026-08-12 03:55:04.451 UTC [31208] LOG: database system is shut down
2026-08-12T03:55:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53241 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:55:04 dist-test-slave-tm1g krb5kdc[496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:55:04 dist-test-slave-tm1g krb5kdc[496](info): set up 2 sockets
Aug 12 03:55:04 dist-test-slave-tm1g krb5kdc[496](info): commencing operation
krb5kdc: starting...
W20260812 03:55:05.681712 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:55:05 dist-test-slave-tm1g krb5kdc[496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:55:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:55:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:55:05.827688 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:38755
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:40317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:38755,127.17.54.253:46855,127.17.54.252:37213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:05.914080 511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:05.914318 511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:05.914345 511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:05.915536 511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:05.915582 511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:05.915597 511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:05.915611 511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:05.915623 511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:05.917228 511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:38755,127.17.54.253:46855,127.17.54.252:37213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:38755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:05.917557 511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:05.917824 511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:05.923597 517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:05.923619 519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:05.923954 511 server_base.cc:1034] running on GCE node
W20260812 03:55:05.923650 516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:05.924304 511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:05.924695 511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:05.925896 511 hybrid_clock.cc:648] HybridClock initialized: now 1786506905925811 us; error 108 us; skew 500 ppm
Aug 12 03:55:05 dist-test-slave-tm1g krb5kdc[496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:05.928673 511 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:55:05.929193 511 webserver.cc:460] Webserver started at http://127.17.54.254:34225/ using document root <none> and password file <none>
I20260812 03:55:05.929469 511 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:05.929518 511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:05.929659 511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:05.930924 511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "7d4cc31e8acd4c1993e5d34394e65be1"
format_stamp: "Formatted at 2026-08-12 03:55:05 on dist-test-slave-tm1g"
server_key: "e3a230057b02bb7d5105bf17ca0849df"
server_key_iv: "d8d2d6b3f2db4a83e80412457f989d70"
server_key_version: "encryptionkey@0"
I20260812 03:55:05.931347 511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "7d4cc31e8acd4c1993e5d34394e65be1"
format_stamp: "Formatted at 2026-08-12 03:55:05 on dist-test-slave-tm1g"
server_key: "e3a230057b02bb7d5105bf17ca0849df"
server_key_iv: "d8d2d6b3f2db4a83e80412457f989d70"
server_key_version: "encryptionkey@0"
I20260812 03:55:05.932802 511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:55:05.933678 526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:05.933892 511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:05.933993 511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "7d4cc31e8acd4c1993e5d34394e65be1"
format_stamp: "Formatted at 2026-08-12 03:55:05 on dist-test-slave-tm1g"
server_key: "e3a230057b02bb7d5105bf17ca0849df"
server_key_iv: "d8d2d6b3f2db4a83e80412457f989d70"
server_key_version: "encryptionkey@0"
I20260812 03:55:05.934073 511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:05.952065 511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:05.954283 511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:05.954536 511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:05.958909 511 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:38755
I20260812 03:55:05.958969 587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:38755 every 8 connection(s)
I20260812 03:55:05.959355 511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:55:05.960280 588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:05.961921 588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:55:05.964244 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 511
I20260812 03:55:05.964403 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:55:05.964601 17627 external_mini_cluster.cc:1428] Setting key c9881a2f512891577b2f953de02263f5
WARNING: no policy specified for kudu/127.17.54.253@KRBTEST.COM; defaulting to no policy
Aug 12 03:55:05 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.253@KRBTEST.COM" created.
I20260812 03:55:05.978718 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:05.962726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:38755 (local address 127.0.0.1:39884)
0812 03:55:05.962918 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:05.962972 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:05.963451 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:05.963776 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:05.963779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:05.964062 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:05.964242 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:05.964246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:05.965800 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:05.965801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:05.966984 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:05.966988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:05.967793 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:05.967794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:05.967873 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:05.973017 (+ 5144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:05.973031 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:05.974998 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:05.976904 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:05.976912 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:05.976914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:05.977208 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:05.977497 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:05.977498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:05.977498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:05.977597 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:05.978147 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:05.978155 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:05.978241 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 03:55:05.978376 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 03:55:05.978532 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"spinlock_wait_cycles":8064,"thread_start_us":98,"threads_started":1}
I20260812 03:55:05.979106 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:05.962740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.17.54.254:38755)
0812 03:55:05.962998 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:05.963002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:05.963047 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:55:05.963625 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:05.963627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:05.963644 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:05.963741 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:05.964368 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:05.965688 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:05.967081 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:05.967577 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:05.967687 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:05.975132 (+ 7445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:05.975157 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:05.975157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:05.975167 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:05.976793 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:05.977292 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:05.977295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:05.977295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:05.977380 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:05.977679 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:05.977680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:05.977681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:05.978043 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:05.978139 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:05.978920 (+ 781us) server_negotiation.cc:299] Negotiation successful
0812 03:55:05.979012 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
I20260812 03:55:05.979871 588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:55:05.980307 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:05.980155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39608)
0812 03:55:05.980214 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:05.980277 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:55:05.980487 530 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
I20260812 03:55:05.980927 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:05.980820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39620)
0812 03:55:05.980867 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:05.980901 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 03:55:05.981050 588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:46855: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab.
Entry for principal kudu/127.17.54.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab.
I20260812 03:55:05.997507 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.253:46855
--webserver_interface=127.17.54.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab
--principal=kudu/127.17.54.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:40317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:38755,127.17.54.253:46855,127.17.54.252:37213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:06.030443 588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } attempt: 1
I20260812 03:55:06.031091 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.030882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39636)
0812 03:55:06.031003 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.031056 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 03:55:06.031589 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.031483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39644)
0812 03:55:06.031533 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.031567 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 03:55:06.031738 588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:46855: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
W20260812 03:55:06.082470 595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:06.082685 595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:06.082759 595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:06.083957 595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:06.084007 595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:06.084021 595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:06.084034 595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:06.084046 595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:06.085618 595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:38755,127.17.54.253:46855,127.17.54.252:37213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab
--principal=kudu/127.17.54.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.253:46855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.54.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:06.085840 595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:06.086040 595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:55:06.091457 595 server_base.cc:1034] running on GCE node
W20260812 03:55:06.091365 601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:06.091408 600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:06.091365 603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:06.091873 595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:06.092198 595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:06.093432 595 hybrid_clock.cc:648] HybridClock initialized: now 1786506906093404 us; error 41 us; skew 500 ppm
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:06.096478 595 init.cc:377] Logged in from keytab as kudu/127.17.54.253@KRBTEST.COM (short username kudu)
I20260812 03:55:06.096962 595 webserver.cc:460] Webserver started at http://127.17.54.253:38599/ using document root <none> and password file <none>
I20260812 03:55:06.097254 595 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:06.097313 595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:06.097448 595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:06.098779 595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/data/instance:
uuid: "25daf00facb94ad1bce2a761dfc72651"
format_stamp: "Formatted at 2026-08-12 03:55:06 on dist-test-slave-tm1g"
server_key: "8c71095e9a98f0beef5db0c44e9770be"
server_key_iv: "b2ae30490e2c72073cee07b46ed1a97f"
server_key_version: "encryptionkey@0"
I20260812 03:55:06.099189 595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/wal/instance:
uuid: "25daf00facb94ad1bce2a761dfc72651"
format_stamp: "Formatted at 2026-08-12 03:55:06 on dist-test-slave-tm1g"
server_key: "8c71095e9a98f0beef5db0c44e9770be"
server_key_iv: "b2ae30490e2c72073cee07b46ed1a97f"
server_key_version: "encryptionkey@0"
I20260812 03:55:06.099926 588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } attempt: 2
I20260812 03:55:06.100433 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.100260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39654)
0812 03:55:06.100335 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.100379 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:55:06.100931 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.100773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39662)
0812 03:55:06.100851 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.100902 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 03:55:06.101161 595 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:55:06.102121 610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260812 03:55:06.102193 588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:46855: Network error: Client connection negotiation failed: client connection to 127.17.54.253:46855: connect: Connection refused (error 111)
I20260812 03:55:06.102288 595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:06.102403 595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/wal
uuid: "25daf00facb94ad1bce2a761dfc72651"
format_stamp: "Formatted at 2026-08-12 03:55:06 on dist-test-slave-tm1g"
server_key: "8c71095e9a98f0beef5db0c44e9770be"
server_key_iv: "b2ae30490e2c72073cee07b46ed1a97f"
server_key_version: "encryptionkey@0"
I20260812 03:55:06.102514 595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:06.120636 595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:06.123109 595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:06.123286 595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:06.127590 595 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.253:46855
I20260812 03:55:06.127719 671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.253:46855 every 8 connection(s)
I20260812 03:55:06.128047 595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
I20260812 03:55:06.128999 672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:06.130779 672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:55:06.133682 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 595
I20260812 03:55:06.133817 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-1/wal/instance
I20260812 03:55:06.134013 17627 external_mini_cluster.cc:1428] Setting key a65b2374b0b2da94c5779aee64bd5a94
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:06.141922 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.131578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39898 (local address 127.17.54.254:38755)
0812 03:55:06.131655 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.131657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.131901 (+ 244us) server_negotiation.cc:407] Connection header received
0812 03:55:06.132347 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.132348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.132361 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.132414 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.133030 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.133533 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.134836 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.135084 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.135167 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.138276 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.138277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.138277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.138285 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.140063 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.140689 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.140690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.140691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.140759 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.141131 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.141133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.141133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.141262 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.141327 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.141730 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.141842 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:55:06.142222 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.131547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:38755 (local address 127.0.0.1:39898)
0812 03:55:06.131767 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.131809 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:06.132265 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:06.132487 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:06.132491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:06.132748 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:06.132955 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.132959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.133639 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.133640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.134738 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.134741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.135176 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.135177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.135316 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.135893 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:06.135904 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:06.138141 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:06.140168 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.140179 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.140182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.140588 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.140841 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.140843 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.140844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.141032 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.141346 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:06.141353 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:06.141552 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 03:55:06.141691 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 03:55:06.141902 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":111,"threads_started":1}
I20260812 03:55:06.143018 672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.54.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.252@KRBTEST.COM" created.
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:55:06.151703 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.143348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39670)
0812 03:55:06.143661 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.143675 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:06.143728 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:06.143939 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:06.143941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:06.144101 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:06.144171 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.144173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.145210 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.145211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.145856 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.145859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.146432 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.146433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.146495 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.146948 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:06.146952 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:06.148490 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:06.150482 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.150483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.150484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.150729 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.150938 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.150938 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.150939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.151013 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.151300 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:06.151301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:06.151340 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:55:06.151407 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:55:06.151524 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293}
I20260812 03:55:06.151723 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.143439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.17.54.253:46855)
0812 03:55:06.143693 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.143698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.143731 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:55:06.143814 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.143816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.143829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.143909 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.144300 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.145113 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.145975 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.146279 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.146428 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.148582 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.148591 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.148591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.148598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.150400 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.150814 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.150817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.150817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.150870 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.151086 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.151087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.151087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.151225 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.151323 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.151437 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.151576 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":218,"thread_start_us":150,"threads_started":1}
I20260812 03:55:06.152587 672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:55:06.152987 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.152827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59806)
0812 03:55:06.152889 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.152950 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 03:55:06.153154 613 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
I20260812 03:55:06.153707 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.153572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59820)
0812 03:55:06.153634 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.153670 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:55:06.153883 672 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37213: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab.
Entry for principal kudu/127.17.54.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab.
I20260812 03:55:06.166909 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.252:37213
--webserver_interface=127.17.54.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab
--principal=kudu/127.17.54.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:40317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:38755,127.17.54.253:46855,127.17.54.252:37213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:06.193380 588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } attempt: 3
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:55:06.201691 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.194031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.17.54.253:46855)
0812 03:55:06.194104 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.194107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.194120 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:55:06.194158 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.194160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.194170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.194245 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.194700 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.195272 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.196059 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.196305 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.196390 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.198786 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.198788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.198788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.198796 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.200482 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.200914 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.200915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.200915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.200972 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.201179 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.201179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.201179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.201290 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.201395 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.201551 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.201621 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:55:06.201738 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.193869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39680)
0812 03:55:06.193964 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.193980 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:06.194075 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:06.194250 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:06.194251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:06.194512 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:06.194596 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.194599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.195368 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.195369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.195961 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.195963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.196400 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.196402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.196496 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.197036 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:06.197041 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:06.198658 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:06.200576 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.200578 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.200578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.200815 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.201043 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.201043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.201044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.201117 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.201392 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:06.201393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:06.201448 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:55:06.201523 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:55:06.201633 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 03:55:06.202236 588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:55:06.202675 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.202550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59832)
0812 03:55:06.202610 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.202648 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:55:06.203102 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.203001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59838)
0812 03:55:06.203032 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.203080 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 03:55:06.203190 588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37213: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
I20260812 03:55:06.203101 672 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } attempt: 1
I20260812 03:55:06.203512 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.203368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59842)
0812 03:55:06.203457 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.203489 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 03:55:06.203941 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.203877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59856)
0812 03:55:06.203902 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.203925 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 03:55:06.204052 672 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37213: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
I20260812 03:55:06.234377 588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } attempt: 1
I20260812 03:55:06.235272 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.234961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59864)
0812 03:55:06.235176 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.235233 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260812 03:55:06.236238 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.236039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59866)
0812 03:55:06.236099 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.236135 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:55:06.236555 588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37213: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
W20260812 03:55:06.255517 679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:06.255729 679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:06.255751 679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:06.256934 679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:06.256978 679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:06.256991 679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:06.257004 679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:06.257016 679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:06.258646 679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:40317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:38755,127.17.54.253:46855,127.17.54.252:37213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab
--principal=kudu/127.17.54.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.252:37213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.54.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:06.258927 679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:06.259174 679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:06.264353 685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:06.264360 687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:06.264360 689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:06.264766 679 server_base.cc:1034] running on GCE node
I20260812 03:55:06.265056 679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:06.265393 679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:06.266572 679 hybrid_clock.cc:648] HybridClock initialized: now 1786506906266558 us; error 33 us; skew 500 ppm
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:06.269229 679 init.cc:377] Logged in from keytab as kudu/127.17.54.252@KRBTEST.COM (short username kudu)
I20260812 03:55:06.269716 679 webserver.cc:460] Webserver started at http://127.17.54.252:46565/ using document root <none> and password file <none>
I20260812 03:55:06.269979 679 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:06.270036 679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:06.270217 679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:06.271391 679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/data/instance:
uuid: "d2422c85649046888a94372d242363b7"
format_stamp: "Formatted at 2026-08-12 03:55:06 on dist-test-slave-tm1g"
server_key: "0c267d714b0a8bb71ee5a46cee8b99ba"
server_key_iv: "cab8bffbcc3044b75bc9be13606c5913"
server_key_version: "encryptionkey@0"
I20260812 03:55:06.271780 679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/wal/instance:
uuid: "d2422c85649046888a94372d242363b7"
format_stamp: "Formatted at 2026-08-12 03:55:06 on dist-test-slave-tm1g"
server_key: "0c267d714b0a8bb71ee5a46cee8b99ba"
server_key_iv: "cab8bffbcc3044b75bc9be13606c5913"
server_key_version: "encryptionkey@0"
I20260812 03:55:06.272229 672 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } attempt: 2
I20260812 03:55:06.272764 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.272619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59868)
0812 03:55:06.272693 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.272734 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 03:55:06.273275 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.273107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59872)
0812 03:55:06.273208 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.273245 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260812 03:55:06.273432 672 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37213: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37213: connect: Connection refused (error 111)
I20260812 03:55:06.273569 679 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 03:55:06.274478 696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:06.274681 679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:06.274787 679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/wal
uuid: "d2422c85649046888a94372d242363b7"
format_stamp: "Formatted at 2026-08-12 03:55:06 on dist-test-slave-tm1g"
server_key: "0c267d714b0a8bb71ee5a46cee8b99ba"
server_key_iv: "cab8bffbcc3044b75bc9be13606c5913"
server_key_version: "encryptionkey@0"
I20260812 03:55:06.274876 679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:06.291035 679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:06.293454 679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:06.293639 679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:06.298018 679 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.252:37213
I20260812 03:55:06.298097 755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.252:37213 every 8 connection(s)
I20260812 03:55:06.298517 679 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
I20260812 03:55:06.299474 756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:06.301386 756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:55:06.303124 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 679
I20260812 03:55:06.303236 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506741478267-17627-0/minicluster-data/master-2/wal/instance
I20260812 03:55:06.303453 17627 external_mini_cluster.cc:1428] Setting key 260c575b6120a19d34cf8e46c4a1b390
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
I20260812 03:55:06.311808 588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } attempt: 2
I20260812 03:55:06.313047 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.302150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39908 (local address 127.17.54.254:38755)
0812 03:55:06.302239 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.302241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.302455 (+ 214us) server_negotiation.cc:407] Connection header received
0812 03:55:06.302991 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.302993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.303007 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.303097 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.304051 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.304655 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.305972 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.306732 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.306803 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.309324 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.309326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.309326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.309335 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.311195 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.311811 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.311812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.311812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.311902 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.312235 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.312236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.312236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.312389 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.312487 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.312859 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.312950 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:55:06.313279 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.302093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:38755 (local address 127.0.0.1:39908)
0812 03:55:06.302285 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.302368 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:06.302881 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:06.303227 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:06.303231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:06.303603 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:06.303880 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.303887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.304754 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.304755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.305836 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.305847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.306819 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.306821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.306959 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.307529 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:06.307540 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:06.309226 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:06.311299 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.311305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.311307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.311688 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.311988 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.311989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.311989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.312144 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.312541 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:06.312549 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:06.312723 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 03:55:06.312838 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:55:06.313011 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":108,"threads_started":1}
I20260812 03:55:06.313436 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.303929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.17.54.252:37213)
0812 03:55:06.304140 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.304144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.304162 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:55:06.304182 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.304184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.304203 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.304287 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.305005 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.305752 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.306664 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.307073 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.307124 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.309969 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.309981 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.309985 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.309993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.311880 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.312388 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.312391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.312392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.312460 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.312671 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.312672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.312672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.312873 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.313030 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.313257 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.313345 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":105,"threads_started":1}
I20260812 03:55:06.313863 756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:06.326771 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.314187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:46855 (local address 127.0.0.1:39690)
0812 03:55:06.314642 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.314653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:06.314723 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:06.314909 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:06.314910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:06.315061 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:06.315119 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.315121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.316092 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.316093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.316974 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.316976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.317532 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.317533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.317678 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.318107 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:06.318113 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:06.319626 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:06.325271 (+ 5645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.325272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.325273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.325526 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.325787 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.325788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.325788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.325840 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.326184 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:06.326185 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:06.326482 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 03:55:06.326536 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:55:06.326645 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440}
I20260812 03:55:06.327271 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.314289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.17.54.253:46855)
0812 03:55:06.314565 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.314567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.314697 (+ 130us) server_negotiation.cc:407] Connection header received
0812 03:55:06.314796 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.314797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.314808 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.314839 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.315217 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.315973 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.317085 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.317471 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.317535 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.319741 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.319742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.319743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.319751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.324139 (+ 4388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.325628 (+ 1489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.325629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.325629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.325701 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.325914 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.325915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.325916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.326071 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.326146 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.327079 (+ 933us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.327169 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260812 03:55:06.327651 756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:55:06.327932 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.312118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59890)
0812 03:55:06.312178 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.312188 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:06.312227 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:06.312498 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:06.312499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:06.312651 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:06.312697 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.312698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.314502 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.314503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.315327 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.315329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.315835 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.315836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.315895 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.316487 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:06.316493 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:06.318208 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:06.326377 (+ 8169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.326379 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.326380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.326663 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.326890 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.326891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.326891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.326965 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.327619 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:06.327620 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:06.327672 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:55:06.327723 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:55:06.327829 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:55:06.328392 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.312188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.17.54.252:37213)
0812 03:55:06.312367 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.312369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.312375 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:55:06.312391 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.312392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.312402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.312474 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.312778 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.314401 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.315430 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.315741 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.316040 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.318362 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.318364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.318364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.318373 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.326261 (+ 7888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.326752 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.326752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.326753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.326820 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.327384 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.327386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.327386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.327535 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.328116 (+ 581us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.328155 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.328247 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260812 03:55:06.328452 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.314602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.17.54.254:38755)
0812 03:55:06.314641 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.314642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.314649 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:55:06.314691 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.314692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.314701 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.314774 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.315458 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.318630 (+ 3172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.319743 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.320066 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.320150 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.325203 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.325205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.325206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.325214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.327027 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.327435 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.327436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.327436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.327491 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.327704 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.327705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.327705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.327827 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.328018 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.328260 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.328359 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:55:06.330184 588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1: Bootstrap starting.
I20260812 03:55:06.331032 588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:06.331359 588 log.cc:824] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:06.332365 588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1: No bootstrap required, opened a new log
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:55:06.334584 588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.334741 588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:06.334769 588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4cc31e8acd4c1993e5d34394e65be1, State: Initialized, Role: FOLLOWER
I20260812 03:55:06.334925 588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.335239 767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.335318 588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:06.335397 767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:55:06.336917 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.328004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.17.54.252:37213)
0812 03:55:06.328041 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.328043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.328325 (+ 282us) server_negotiation.cc:407] Connection header received
0812 03:55:06.328341 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.328342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.328351 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.328388 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.329146 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.329825 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.330775 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.331002 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.331081 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.333228 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.333229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.333229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.333237 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.335469 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.335861 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.335862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.335862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.335925 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.336146 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.336146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.336147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.336366 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.336418 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.336662 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.336813 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":17}
I20260812 03:55:06.336918 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.327975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59900)
0812 03:55:06.328214 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.328227 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:06.328275 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:06.328431 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:06.328431 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:06.329024 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:06.329067 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.329069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.329914 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.329915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.330697 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.330700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.331077 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.331077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.331126 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.331564 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:06.331570 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:06.333140 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:06.335555 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.335556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.335557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.335776 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.336025 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.336025 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.336025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.336087 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.336461 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:06.336462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:06.336500 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:55:06.336620 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 03:55:06.336740 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260812 03:55:06.338274 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.329080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.17.54.253:46855)
0812 03:55:06.329117 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.329118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.329126 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:55:06.329190 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.329191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.329200 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.329253 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.329763 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.330511 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.331613 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.331826 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.331859 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.334641 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.334642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.334643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.334650 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.336912 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.337366 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.337367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.337368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.337437 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.337655 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.337657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.337657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.337818 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.338035 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.338076 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.338155 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 03:55:06.338986 779 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:55:06.339047 779 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:55:06.339564 756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7: Bootstrap starting.
I20260812 03:55:06.340263 756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:06.340572 756 log.cc:824] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:06.341470 756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7: No bootstrap required, opened a new log
I20260812 03:55:06.342198 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:55:06.343793 756 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.343964 756 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:06.344008 756 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2422c85649046888a94372d242363b7, State: Initialized, Role: FOLLOWER
I20260812 03:55:06.344132 756 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.344415 786 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.344481 756 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:06.344563 786 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:55:06.342589 511 master_runner.cc:186] Error getting master registration for 127.17.54.253:46855: OK, instance_id { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" instance_seqno: 1786506906126239 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:55:06.344998 511 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 03:55:06.348104 797 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:55:06.348273 797 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:55:06.349457 679 master_runner.cc:186] Error getting master registration for 127.17.54.253:46855: OK, instance_id { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" instance_seqno: 1786506906126239 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:55:06.353088 679 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:55:06.359777 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.346887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.17.54.254:38755)
0812 03:55:06.346957 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.346959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.347270 (+ 311us) server_negotiation.cc:407] Connection header received
0812 03:55:06.347377 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.347379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.347392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.347423 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.348343 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.349220 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.354347 (+ 5127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.354699 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.354749 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.356569 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.356571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.356571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.356580 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.358253 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.358861 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.358862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.358863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.358927 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.359132 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.359133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.359133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.359274 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.359364 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.359584 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.359674 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:55:06.362043 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.346594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.17.54.252:37213)
0812 03:55:06.346641 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.346643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.347675 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 03:55:06.348835 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.348837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.348848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.348880 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.351841 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.352650 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.354556 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.354867 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.354912 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.357211 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.357213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.357213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.357221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.359351 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.361051 (+ 1700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.361052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.361052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.361117 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.361378 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.361379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.361380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.361532 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.361623 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.361854 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.361946 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:55:06.362095 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.347028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.17.54.253:46855)
0812 03:55:06.347080 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.347082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.347553 (+ 471us) server_negotiation.cc:407] Connection header received
0812 03:55:06.347805 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.347807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.347823 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.347866 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.349835 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.350726 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.353776 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.354076 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.354124 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.357542 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.357544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.357544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.357553 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.360117 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.361027 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.361028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.361029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.361101 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.361349 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.361350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.361350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.361592 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.361684 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.361905 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.361996 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:55:06.364596 672 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } attempt: 3
Aug 12 03:55:06 dist-test-slave-tm1g krb5kdc[496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
I20260812 03:55:06.373179 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.364874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37213 (local address 127.0.0.1:59914)
0812 03:55:06.364932 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:06.364942 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:06.364980 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:06.365249 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:06.365251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:06.365437 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:06.365490 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.365491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.366450 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.366451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.367277 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:06.367280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.367769 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.367771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:06.367835 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.368366 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:06.368372 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:06.370017 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:06.371969 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.371970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.371971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.372215 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.372446 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:06.372447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:06.372447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:06.372528 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:06.372823 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:06.372824 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:06.372877 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:55:06.372953 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:55:06.373065 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 03:55:06.373185 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:06.365052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59914 (local address 127.17.54.252:37213)
0812 03:55:06.365115 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:06.365117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:06.365126 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:55:06.365149 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:06.365150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:06.365159 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:06.365225 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:06.365582 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.366364 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.367369 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:06.367677 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:06.367760 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:06.370107 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:06.370108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:06.370108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:06.370116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:06.371869 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.372300 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.372301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.372301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.372366 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:06.372602 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:06.372603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:06.372604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:06.372748 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:06.372818 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:06.372992 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 03:55:06.373084 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:55:06.375346 672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651: Bootstrap starting.
I20260812 03:55:06.376156 672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:06.376463 672 log.cc:824] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:06.377496 672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651: No bootstrap required, opened a new log
I20260812 03:55:06.379709 672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.379863 672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:06.379899 672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25daf00facb94ad1bce2a761dfc72651, State: Initialized, Role: FOLLOWER
I20260812 03:55:06.380033 672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.380373 672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:06.380342 804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.380648 804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:55:06.382148 767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:55:06.382265 767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.382519 767 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25daf00facb94ad1bce2a761dfc72651 (127.17.54.253:46855), d2422c85649046888a94372d242363b7 (127.17.54.252:37213)
I20260812 03:55:06.382877 728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2422c85649046888a94372d242363b7" is_pre_election: true
I20260812 03:55:06.383047 728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4cc31e8acd4c1993e5d34394e65be1 in term 0.
I20260812 03:55:06.383317 529 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d4cc31e8acd4c1993e5d34394e65be1, d2422c85649046888a94372d242363b7; no voters:
I20260812 03:55:06.383365 640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25daf00facb94ad1bce2a761dfc72651" is_pre_election: true
I20260812 03:55:06.383477 640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4cc31e8acd4c1993e5d34394e65be1 in term 0.
I20260812 03:55:06.383463 767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:55:06.383512 767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:55:06.383535 767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:06.384464 767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.384593 767 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [CANDIDATE]: Term 1 election: Requested vote from peers 25daf00facb94ad1bce2a761dfc72651 (127.17.54.253:46855), d2422c85649046888a94372d242363b7 (127.17.54.252:37213)
I20260812 03:55:06.384785 640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25daf00facb94ad1bce2a761dfc72651"
I20260812 03:55:06.384814 727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2422c85649046888a94372d242363b7"
I20260812 03:55:06.384877 640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:06.384887 727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:06.385792 640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d4cc31e8acd4c1993e5d34394e65be1 in term 1.
I20260812 03:55:06.385792 727 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d4cc31e8acd4c1993e5d34394e65be1 in term 1.
I20260812 03:55:06.385965 530 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25daf00facb94ad1bce2a761dfc72651, 7d4cc31e8acd4c1993e5d34394e65be1; no voters:
I20260812 03:55:06.386296 767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:06.386536 767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [term 1 LEADER]: Becoming Leader. State: Replica: 7d4cc31e8acd4c1993e5d34394e65be1, State: Running, Role: LEADER
I20260812 03:55:06.386641 767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } }
I20260812 03:55:06.387073 767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4cc31e8acd4c1993e5d34394e65be1. Latest consensus state: current_term: 1 leader_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.387152 767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:06.387311 595 master_runner.cc:232] Existing masters have differing terms: 0 vs 1
I20260812 03:55:06.387392 595 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 03:55:06.387810 817 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:55:06.387861 817 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:55:06.388578 820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:55:06.388777 820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:55:06.390419 640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [term 1 FOLLOWER]: Refusing update from remote peer 7d4cc31e8acd4c1993e5d34394e65be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:06.390710 728 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [term 1 FOLLOWER]: Refusing update from remote peer 7d4cc31e8acd4c1993e5d34394e65be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:06.391024 767 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:06.391178 767 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:06.392882 804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4cc31e8acd4c1993e5d34394e65be1. Latest consensus state: current_term: 1 leader_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.392982 804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:55:06.393055 786 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4cc31e8acd4c1993e5d34394e65be1. Latest consensus state: current_term: 1 leader_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.393138 786 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:55:06.393632 767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.393719 767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:06.394421 786 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.394508 786 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:55:06.394557 822 mvcc.cc:205] Tried to move back new op lower bound from 7317532288572186624 to 7317532288560087040. Current Snapshot: MvccSnapshot[applied={T|T < 7317532288572186624}]
I20260812 03:55:06.398835 820 catalog_manager.cc:1347] Generated new cluster ID: adb5f04983ae47ef89a28e737b929c12
I20260812 03:55:06.398901 820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:55:06.399338 824 mvcc.cc:205] Tried to move back new op lower bound from 7317532288572186624 to 7317532288560087040. Current Snapshot: MvccSnapshot[applied={T|T < 7317532288572186624}]
I20260812 03:55:06.400379 767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.400497 767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:06.400661 804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d4cc31e8acd4c1993e5d34394e65be1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38755 } } peers { permanent_uuid: "25daf00facb94ad1bce2a761dfc72651" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 46855 } } peers { permanent_uuid: "d2422c85649046888a94372d242363b7" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37213 } } }
I20260812 03:55:06.400771 804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:55:06.400825 823 mvcc.cc:205] Tried to move back new op lower bound from 7317532288572186624 to 7317532288560087040. Current Snapshot: MvccSnapshot[applied={T|T < 7317532288572186624}]
I20260812 03:55:06.424391 820 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:55:06.425128 820 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:55:06.434540 820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d4cc31e8acd4c1993e5d34394e65be1: Generated new TSK 0
I20260812 03:55:06.434798 820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:55:07.348843 797 catalog_manager.cc:1259] Loaded cluster ID: adb5f04983ae47ef89a28e737b929c12
I20260812 03:55:07.348978 797 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7: loading cluster ID for follower catalog manager: success
I20260812 03:55:07.349802 797 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7: acquiring CA information for follower catalog manager: success
I20260812 03:55:07.350106 797 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d2422c85649046888a94372d242363b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:55:07.388957 817 catalog_manager.cc:1259] Loaded cluster ID: adb5f04983ae47ef89a28e737b929c12
I20260812 03:55:07.389056 817 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651: loading cluster ID for follower catalog manager: success
I20260812 03:55:07.389914 817 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651: acquiring CA information for follower catalog manager: success
I20260812 03:55:07.390216 817 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25daf00facb94ad1bce2a761dfc72651: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:55:07.491557 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 511
I20260812 03:55:07.499600 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 595
I20260812 03:55:07.508459 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 679
2026-08-12T03:55:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2947 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:55:07 dist-test-slave-tm1g krb5kdc[839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:55:07 dist-test-slave-tm1g krb5kdc[839](info): set up 2 sockets
Aug 12 03:55:07 dist-test-slave-tm1g krb5kdc[839](info): commencing operation
krb5kdc: starting...
W20260812 03:55:08.655208 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:55:08 dist-test-slave-tm1g krb5kdc[839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:55:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:55:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:55:08.797287 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39151
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:39151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:08.887706 852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:08.887934 852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:08.887959 852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:08.889171 852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:08.889223 852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:08.889238 852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:08.889251 852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:08.889263 852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:08.890848 852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:39151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:08.891129 852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:08.891389 852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:08.897715 861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:08.897715 858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:08.897727 857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:08.898191 852 server_base.cc:1034] running on GCE node
I20260812 03:55:08.898545 852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:08.898842 852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:08.900002 852 hybrid_clock.cc:648] HybridClock initialized: now 1786506908899986 us; error 30 us; skew 500 ppm
Aug 12 03:55:08 dist-test-slave-tm1g krb5kdc[839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506908, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:08.902905 852 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:55:08.903522 852 webserver.cc:460] Webserver started at http://127.17.54.254:43939/ using document root <none> and password file <none>
I20260812 03:55:08.903771 852 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:08.903841 852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:08.903996 852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:08.905115 852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "a43d80c0d07d41ca98e4ecddf791ce74"
format_stamp: "Formatted at 2026-08-12 03:55:08 on dist-test-slave-tm1g"
server_key: "bce05769c7bd48198bf5037f333c6eeb"
server_key_iv: "be5756a0a02af385dca0649671809961"
server_key_version: "encryptionkey@0"
I20260812 03:55:08.905566 852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "a43d80c0d07d41ca98e4ecddf791ce74"
format_stamp: "Formatted at 2026-08-12 03:55:08 on dist-test-slave-tm1g"
server_key: "bce05769c7bd48198bf5037f333c6eeb"
server_key_iv: "be5756a0a02af385dca0649671809961"
server_key_version: "encryptionkey@0"
I20260812 03:55:08.907130 852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:55:08.908032 869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:08.908242 852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:08.908380 852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "a43d80c0d07d41ca98e4ecddf791ce74"
format_stamp: "Formatted at 2026-08-12 03:55:08 on dist-test-slave-tm1g"
server_key: "bce05769c7bd48198bf5037f333c6eeb"
server_key_iv: "be5756a0a02af385dca0649671809961"
server_key_version: "encryptionkey@0"
I20260812 03:55:08.908466 852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:08.922592 852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:08.924846 852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:08.925031 852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:08.929596 852 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:39151
I20260812 03:55:08.929684 933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:39151 every 8 connection(s)
I20260812 03:55:08.930095 852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:55:08.931133 934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:08.933833 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 852
I20260812 03:55:08.933842 934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74: Bootstrap starting.
I20260812 03:55:08.933939 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:55:08.934165 17627 external_mini_cluster.cc:1428] Setting key 96ca7d43ed976233a1df2955191644c1
I20260812 03:55:08.934723 934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:08.935113 934 log.cc:824] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:08.936015 934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74: No bootstrap required, opened a new log
I20260812 03:55:08.938385 934 raft_consensus.cc:348] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43d80c0d07d41ca98e4ecddf791ce74" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39151 } }
I20260812 03:55:08.938614 934 raft_consensus.cc:374] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:08.938647 934 raft_consensus.cc:729] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43d80c0d07d41ca98e4ecddf791ce74, State: Initialized, Role: FOLLOWER
I20260812 03:55:08.938781 934 consensus_queue.cc:260] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43d80c0d07d41ca98e4ecddf791ce74" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39151 } }
I20260812 03:55:08.938858 934 raft_consensus.cc:388] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:55:08.938899 934 raft_consensus.cc:482] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:55:08.938997 934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:08.939694 934 raft_consensus.cc:504] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43d80c0d07d41ca98e4ecddf791ce74" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39151 } }
I20260812 03:55:08.939818 934 leader_election.cc:302] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a43d80c0d07d41ca98e4ecddf791ce74; no voters:
I20260812 03:55:08.939988 934 leader_election.cc:288] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:55:08.940053 939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:08.940234 939 raft_consensus.cc:686] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [term 1 LEADER]: Becoming Leader. State: Replica: a43d80c0d07d41ca98e4ecddf791ce74, State: Running, Role: LEADER
I20260812 03:55:08.940282 934 sys_catalog.cc:564] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:08.940408 939 consensus_queue.cc:237] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43d80c0d07d41ca98e4ecddf791ce74" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39151 } }
I20260812 03:55:08.940872 940 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a43d80c0d07d41ca98e4ecddf791ce74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43d80c0d07d41ca98e4ecddf791ce74" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39151 } } }
I20260812 03:55:08.940858 939 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a43d80c0d07d41ca98e4ecddf791ce74. Latest consensus state: current_term: 1 leader_uuid: "a43d80c0d07d41ca98e4ecddf791ce74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43d80c0d07d41ca98e4ecddf791ce74" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39151 } } }
I20260812 03:55:08.940968 940 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:08.940972 939 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:08.941337 946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:55:08.941936 946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:55:08 dist-test-slave-tm1g krb5kdc[839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:08.944397 946 catalog_manager.cc:1347] Generated new cluster ID: 813cf8e71b994a3dabb9b22bca15dd4f
I20260812 03:55:08.944497 946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:55:08.947101 937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:08.934795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.17.54.254:39151)
0812 03:55:08.935023 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:08.935031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:08.935058 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:55:08.935748 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:08.935757 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:08.935810 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:08.935946 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:08.936814 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:08.937763 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:08.938930 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:08.939440 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:08.939528 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:08.942822 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:08.942826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:08.942826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:08.942839 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:08.945008 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:08.945480 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:08.945483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:08.945484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:08.945588 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:08.945823 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:08.945823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:08.945824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:08.946116 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:08.946266 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:08.946688 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 03:55:08.946845 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":118,"threads_started":1}
I20260812 03:55:08.948112 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:55:08.952230 946 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:55:08.952785 946 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:55:08.958364 946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a43d80c0d07d41ca98e4ecddf791ce74: Generated new TSK 0
I20260812 03:55:08.958575 946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:55:09.002502 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 852
2026-08-12T03:55:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1493 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:55:09 dist-test-slave-tm1g krb5kdc[971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:55:09 dist-test-slave-tm1g krb5kdc[971](info): set up 2 sockets
Aug 12 03:55:09 dist-test-slave-tm1g krb5kdc[971](info): commencing operation
krb5kdc: starting...
W20260812 03:55:10.151175 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:55:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:55:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:55:10.286587 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34375
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:34375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:10.376040 986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:10.376324 986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:10.376366 986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:10.377581 986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:10.377655 986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:10.377678 986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:10.377698 986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:10.377730 986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:10.379382 986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:34375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:10.379684 986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:10.379935 986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:10.385167 992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:10.385159 991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:10.385154 995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:10.385510 986 server_base.cc:1034] running on GCE node
I20260812 03:55:10.385795 986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:10.386094 986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:10.387272 986 hybrid_clock.cc:648] HybridClock initialized: now 1786506910387266 us; error 39 us; skew 500 ppm
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:10.389892 986 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:55:10.390446 986 webserver.cc:460] Webserver started at http://127.17.54.254:33427/ using document root <none> and password file <none>
I20260812 03:55:10.390682 986 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:10.390751 986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:10.390921 986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:10.392393 986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "f77bb8b5ab924caaa8ea3be6532ba53e"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "ca27a020283f1e00feae064cec4e8a63"
server_key_iv: "ed097a5acf4dfc098b99aa6665db3b86"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.392711 986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "f77bb8b5ab924caaa8ea3be6532ba53e"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "ca27a020283f1e00feae064cec4e8a63"
server_key_iv: "ed097a5acf4dfc098b99aa6665db3b86"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.394059 986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:55:10.394958 1002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.395236 986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:10.395324 986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "f77bb8b5ab924caaa8ea3be6532ba53e"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "ca27a020283f1e00feae064cec4e8a63"
server_key_iv: "ed097a5acf4dfc098b99aa6665db3b86"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.395401 986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:10.412607 986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:10.414935 986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:10.415059 986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:10.419225 986 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:34375
I20260812 03:55:10.419288 1062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:34375 every 8 connection(s)
I20260812 03:55:10.419634 986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:55:10.420687 1063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:10.422605 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 986
I20260812 03:55:10.422695 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:55:10.422895 17627 external_mini_cluster.cc:1428] Setting key e00d8a0a0215342ad4842c66c664a049
I20260812 03:55:10.423329 1063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Bootstrap starting.
I20260812 03:55:10.424023 1063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:10.424341 1063 log.cc:824] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:10.425172 1063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: No bootstrap required, opened a new log
I20260812 03:55:10.427117 1063 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:10.427278 1063 raft_consensus.cc:374] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:10.427325 1063 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77bb8b5ab924caaa8ea3be6532ba53e, State: Initialized, Role: FOLLOWER
I20260812 03:55:10.427487 1063 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:10.427589 1063 raft_consensus.cc:388] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:55:10.427639 1063 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:55:10.427692 1063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:10.428448 1063 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:10.428573 1063 leader_election.cc:302] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f77bb8b5ab924caaa8ea3be6532ba53e; no voters:
I20260812 03:55:10.428740 1063 leader_election.cc:288] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:55:10.428785 1068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:10.428953 1068 raft_consensus.cc:686] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 1 LEADER]: Becoming Leader. State: Replica: f77bb8b5ab924caaa8ea3be6532ba53e, State: Running, Role: LEADER
I20260812 03:55:10.429036 1063 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:10.429056 1068 consensus_queue.cc:237] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:10.429462 1069 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } } }
I20260812 03:55:10.429555 1069 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:10.430388 1070 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f77bb8b5ab924caaa8ea3be6532ba53e. Latest consensus state: current_term: 1 leader_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } } }
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:10.430651 1070 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: This master's current role is: LEADER
W20260812 03:55:10.430918 1088 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:55:10.431495 1088 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:55:10.431768 1078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:55:10.432034 1078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:55:10.433619 1078 catalog_manager.cc:1347] Generated new cluster ID: 1ac9e4b1bb484e2da311d19776059dd4
I20260812 03:55:10.433672 1078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:55:10.435617 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.423584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.17.54.254:34375)
0812 03:55:10.423788 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:10.423798 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:10.423943 (+ 145us) server_negotiation.cc:407] Connection header received
0812 03:55:10.424697 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:10.424706 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:10.424771 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:10.424967 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:10.425692 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.426678 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.427822 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.428435 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.428575 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:10.431770 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:10.431775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:10.431775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:10.431787 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:10.433882 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.434293 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.434297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.434297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.434435 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.434651 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.434652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.434652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.434992 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:10.435173 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:10.435246 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:55:10.435395 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260812 03:55:10.449992 1078 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:55:10.450699 1078 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:55:10.454474 1078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Generated new TSK 0
I20260812 03:55:10.454704 1078 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:55:10.493333 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34375
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:10.578816 1094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:10.579070 1094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:10.579113 1094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:10.580289 1094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:10.580367 1094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:10.580425 1094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:55:10.581979 1094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:10.582260 1094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:10.582540 1094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:10.588251 1099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:10.588258 1100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:10.588234 1102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:10.588569 1094 server_base.cc:1034] running on GCE node
I20260812 03:55:10.588786 1094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:10.589000 1094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:10.590188 1094 hybrid_clock.cc:648] HybridClock initialized: now 1786506910590170 us; error 53 us; skew 500 ppm
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:10.592813 1094 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:55:10.593273 1094 webserver.cc:460] Webserver started at http://127.17.54.193:41709/ using document root <none> and password file <none>
I20260812 03:55:10.593561 1094 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:10.593637 1094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:10.593796 1094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:10.594923 1094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "2969f0211eeb4a1e880d11ef509b1113"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "1a970f512a298f9ad561d15dd4fcc961"
server_key_iv: "84f1accea85a03dae29451c5cf2bf43d"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.595284 1094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "2969f0211eeb4a1e880d11ef509b1113"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "1a970f512a298f9ad561d15dd4fcc961"
server_key_iv: "84f1accea85a03dae29451c5cf2bf43d"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.596673 1094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:55:10.597462 1111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.597666 1094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:10.597774 1094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "2969f0211eeb4a1e880d11ef509b1113"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "1a970f512a298f9ad561d15dd4fcc961"
server_key_iv: "84f1accea85a03dae29451c5cf2bf43d"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.597851 1094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:10.627583 1094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:10.629943 1094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:10.630143 1094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:10.630442 1094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:10.630827 1094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:10.630882 1094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.630926 1094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:10.630955 1094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.636759 1094 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:36903
I20260812 03:55:10.636842 1245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:36903 every 8 connection(s)
I20260812 03:55:10.637140 1094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:55:10.640154 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 1094
I20260812 03:55:10.640269 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:55:10.640493 17627 external_mini_cluster.cc:1428] Setting key 30bd257b0003a5b0ff4bfb77fed6e34b
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:10.648365 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.637908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:42755 (local address 127.17.54.254:34375)
0812 03:55:10.637988 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:10.637990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:10.638186 (+ 196us) server_negotiation.cc:407] Connection header received
0812 03:55:10.638700 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:10.638701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:10.638712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:10.638790 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:10.639740 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.640543 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.642074 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.642291 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.642424 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:10.644976 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:10.644978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:10.644979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:10.644989 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:10.646813 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.647262 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.647264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.647264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.647335 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.647597 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.647599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.647599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.647775 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:10.647857 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:10.648131 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 03:55:10.648238 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:55:10.648391 1249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.637879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:42755)
0812 03:55:10.638049 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:10.638093 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:10.638603 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:10.638847 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:10.638850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:10.639430 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:10.639643 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.639647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.640675 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.640676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:10.641952 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.641969 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.642445 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.642447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:10.642612 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:10.643175 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:10.643187 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:10.644876 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:10.646915 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:10.646923 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:10.646925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:10.647181 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:10.647410 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:10.647411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:10.647411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:10.647526 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:10.647847 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:10.647852 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:10.647991 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 03:55:10.648078 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:55:10.648213 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260812 03:55:10.649068 1247 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34375
I20260812 03:55:10.649194 1247 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:10.649394 1247 heartbeater.cc:511] Master 127.17.54.254:34375 requested a full tablet report, sending...
I20260812 03:55:10.649992 1020 ts_manager.cc:194] Registered new tserver with Master: 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903)
I20260812 03:55:10.650688 1020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:42755
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:55:10.674067 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34375
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:10.759027 1252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:10.759302 1252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:10.759344 1252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:10.760543 1252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:10.760627 1252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:10.760672 1252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:55:10.762208 1252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:10.762537 1252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:10.762797 1252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:10.768388 1259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:10.768395 1261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:10.768414 1258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:10.768591 1252 server_base.cc:1034] running on GCE node
I20260812 03:55:10.768882 1252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:10.769155 1252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:10.770406 1252 hybrid_clock.cc:648] HybridClock initialized: now 1786506910770385 us; error 38 us; skew 500 ppm
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:10.773151 1252 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:55:10.773641 1252 webserver.cc:460] Webserver started at http://127.17.54.194:42457/ using document root <none> and password file <none>
I20260812 03:55:10.773886 1252 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:10.773958 1252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:10.774120 1252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:10.775269 1252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "e0150b6327374a9fa5e38591fe5264e1"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "73f9717805581c84cbf32d64c1cca6b6"
server_key_iv: "5a881ad3a1f5e13ebd1b33425f63892b"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.775691 1252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "e0150b6327374a9fa5e38591fe5264e1"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "73f9717805581c84cbf32d64c1cca6b6"
server_key_iv: "5a881ad3a1f5e13ebd1b33425f63892b"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.777104 1252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:55:10.778014 1268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.778268 1252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:10.778411 1252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "e0150b6327374a9fa5e38591fe5264e1"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "73f9717805581c84cbf32d64c1cca6b6"
server_key_iv: "5a881ad3a1f5e13ebd1b33425f63892b"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.778501 1252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:10.786496 1252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:10.788841 1252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:10.789014 1252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:10.789261 1252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:10.789589 1252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:10.789646 1252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.789690 1252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:10.789723 1252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.795873 1252 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:41081
I20260812 03:55:10.795950 1399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:41081 every 8 connection(s)
I20260812 03:55:10.796280 1252 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:55:10.800417 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 1252
I20260812 03:55:10.800611 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:55:10.800835 17627 external_mini_cluster.cc:1428] Setting key 59d35b522f7236aee1d9074eebe68c9c
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:10.807076 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.797090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:47635 (local address 127.17.54.254:34375)
0812 03:55:10.797179 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:10.797181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:10.797363 (+ 182us) server_negotiation.cc:407] Connection header received
0812 03:55:10.797909 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:10.797911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:10.797924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:10.798002 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:10.799062 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.799608 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.800865 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.801145 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.801237 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:10.804044 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:10.804046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:10.804046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:10.804054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:10.805539 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.806012 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.806013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.806014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.806079 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.806379 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.806380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.806380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.806510 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:10.806593 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:10.806877 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 03:55:10.806982 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:55:10.807277 1402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.797042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:47635)
0812 03:55:10.797242 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:10.797285 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:10.797795 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:10.798078 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:10.798081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:10.798764 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:10.798972 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.798976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.799711 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.799712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:10.800761 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.800765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.801245 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.801247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:10.801393 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:10.802038 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:10.802047 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:10.803931 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:10.805647 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:10.805654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:10.805657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:10.805947 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:10.806159 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:10.806159 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:10.806160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:10.806274 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:10.806582 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:10.806589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:10.806737 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 03:55:10.806841 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:55:10.807011 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":120,"threads_started":1}
I20260812 03:55:10.807986 1400 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34375
I20260812 03:55:10.808075 1400 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:10.808279 1400 heartbeater.cc:511] Master 127.17.54.254:34375 requested a full tablet report, sending...
I20260812 03:55:10.808715 1020 ts_manager.cc:194] Registered new tserver with Master: e0150b6327374a9fa5e38591fe5264e1 (127.17.54.194:41081)
I20260812 03:55:10.809115 1020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:47635
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:55:10.834474 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34375
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:10.927847 1405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:10.928121 1405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:10.928165 1405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:10.929673 1405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:10.929821 1405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:10.929973 1405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:55:10.932073 1405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:10.932400 1405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:10.932682 1405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:10.938676 1414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:10.938704 1412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:10.938676 1411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:10.938871 1405 server_base.cc:1034] running on GCE node
I20260812 03:55:10.939157 1405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:10.939441 1405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:10.940600 1405 hybrid_clock.cc:648] HybridClock initialized: now 1786506910940585 us; error 31 us; skew 500 ppm
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:10.943531 1405 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:55:10.944038 1405 webserver.cc:460] Webserver started at http://127.17.54.195:32967/ using document root <none> and password file <none>
I20260812 03:55:10.944279 1405 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:10.944329 1405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:10.944499 1405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:10.945696 1405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "4d77b3a8b3c448f39249216fb32bf7b3"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "4679029687bf61f4ef98df74d697b915"
server_key_iv: "3e79e8f03fb535c3d57371d8817b04da"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.946096 1405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "4d77b3a8b3c448f39249216fb32bf7b3"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "4679029687bf61f4ef98df74d697b915"
server_key_iv: "3e79e8f03fb535c3d57371d8817b04da"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.947616 1405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:10.948559 1423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.948832 1405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:10.948921 1405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "4d77b3a8b3c448f39249216fb32bf7b3"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "4679029687bf61f4ef98df74d697b915"
server_key_iv: "3e79e8f03fb535c3d57371d8817b04da"
server_key_version: "encryptionkey@0"
I20260812 03:55:10.949026 1405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:10.957283 1405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:10.959641 1405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:10.959765 1405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:10.960052 1405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:10.960381 1405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:10.960436 1405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.960479 1405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:10.960507 1405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:10.966485 1405 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:39237
I20260812 03:55:10.966567 1552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:39237 every 8 connection(s)
I20260812 03:55:10.966940 1405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:55:10.970544 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 1405
I20260812 03:55:10.970664 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:55:10.970849 17627 external_mini_cluster.cc:1428] Setting key 6c5328bcad954bdec5b2f55efcbd933f
Aug 12 03:55:10 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:10.976786 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.967783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:36431 (local address 127.17.54.254:34375)
0812 03:55:10.967867 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:10.967869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:10.968078 (+ 209us) server_negotiation.cc:407] Connection header received
0812 03:55:10.968615 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:10.968617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:10.968628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:10.968697 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:10.969641 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.970167 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.971637 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.971925 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.971995 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:10.974158 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:10.974160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:10.974160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:10.974167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:10.975460 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.975868 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.975869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.975870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.975941 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.976163 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.976163 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.976164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.976272 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:10.976328 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:10.976638 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 03:55:10.976707 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:55:10.977067 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.967742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:36431)
0812 03:55:10.967928 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:10.967973 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:10.968489 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:10.968749 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:10.968752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:10.969342 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:10.969542 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.969547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.970273 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.970274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:10.971511 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.971515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.972015 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.972016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:10.972122 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:10.972634 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:10.972642 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:10.974074 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:10.975546 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:10.975553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:10.975555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:10.975787 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:10.976004 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:10.976005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:10.976005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:10.976104 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:10.976367 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:10.976374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:10.976506 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 03:55:10.976611 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:55:10.976754 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260812 03:55:10.977725 1553 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34375
I20260812 03:55:10.977837 1553 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:10.978003 1553 heartbeater.cc:511] Master 127.17.54.254:34375 requested a full tablet report, sending...
I20260812 03:55:10.978509 1020 ts_manager.cc:194] Registered new tserver with Master: 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237)
I20260812 03:55:10.978979 1020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:36431
I20260812 03:55:10.982807 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:55:10.990669 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.984254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.17.54.254:34375)
0812 03:55:10.984332 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:10.984334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:10.984468 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:55:10.984508 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:10.984509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:10.984520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:10.984576 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:10.985154 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.985656 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.986528 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.986727 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.986787 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:10.988149 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:10.988151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:10.988151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:10.988158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:10.989511 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.989851 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.989851 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.989852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.989904 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:10.990093 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:10.990094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:10.990094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:10.990239 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:10.990372 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:10.990478 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 03:55:10.990549 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:55:10.992660 1020 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:55:10.994375 1020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:55:11.001372 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.996971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:41081 (local address 127.0.0.1:39870)
0812 03:55:10.997451 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:10.997461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:10.997516 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:10.998012 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:10.998013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:10.998016 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:10.998053 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.998055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.999290 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.999291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.000273 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.000276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.001016 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.001017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.001098 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.001113 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.001209 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.001268 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":88,"threads_started":1}
I20260812 03:55:11.001616 1570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.997159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44442 (local address 127.17.54.193:36903)
0812 03:55:10.997419 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:10.997424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:10.997434 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:55:10.997478 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:10.997481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:10.997509 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:10.997597 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:10.997781 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.999017 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.000378 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.001107 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.001187 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.001244 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.001479 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.001536 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":89,"threads_started":1}
I20260812 03:55:11.001693 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.996905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.17.54.195:39237)
0812 03:55:10.997387 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:10.997391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:10.997461 (+ 70us) server_negotiation.cc:407] Connection header received
0812 03:55:10.997514 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:10.997516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:10.997532 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:10.997606 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:10.997826 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.998769 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.000503 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.001276 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.001340 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.001387 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.001558 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.001609 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":133,"threads_started":1}
I20260812 03:55:11.001914 1571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.997135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.17.54.194:41081)
0812 03:55:10.997330 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:10.997335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:10.997527 (+ 192us) server_negotiation.cc:407] Connection header received
0812 03:55:10.997906 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:10.997909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:10.997924 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:10.998003 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:10.998114 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.999194 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.000392 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.000889 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.001000 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.001064 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.001765 (+ 701us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.001824 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260812 03:55:11.001516 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.996763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36903 (local address 127.0.0.1:44442)
0812 03:55:10.997131 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:10.997177 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:10.997244 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:10.997618 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:10.997622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:10.997624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:10.997698 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.997700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.999120 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.999121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.000282 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.000285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.001195 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.001196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.001290 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.001293 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.001352 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.001393 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":23,"thread_start_us":92,"threads_started":1}
I20260812 03:55:11.001619 1566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:10.996764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:39237 (local address 127.0.0.1:38336)
0812 03:55:10.997357 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:10.997369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:10.997451 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:10.997683 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:10.997683 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:10.997686 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:10.997745 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:10.997746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:10.998893 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:10.998894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.000404 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.000406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.001343 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.001344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.001409 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.001413 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.001474 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.001515 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":65,"thread_start_us":56,"threads_started":1}
I20260812 03:55:11.002771 1170 tablet_service.cc:1467] Processing CreateTablet for tablet 5f6ec789f8a14aef9ed60cf6681ce254 (DEFAULT_TABLE table=test-table [id=40683552b2b2461e86c00e07ccf329b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:11.002802 1326 tablet_service.cc:1467] Processing CreateTablet for tablet 5f6ec789f8a14aef9ed60cf6681ce254 (DEFAULT_TABLE table=test-table [id=40683552b2b2461e86c00e07ccf329b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:11.002854 1480 tablet_service.cc:1467] Processing CreateTablet for tablet 5f6ec789f8a14aef9ed60cf6681ce254 (DEFAULT_TABLE table=test-table [id=40683552b2b2461e86c00e07ccf329b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:11.003166 1170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f6ec789f8a14aef9ed60cf6681ce254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:11.003166 1480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f6ec789f8a14aef9ed60cf6681ce254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:11.003167 1326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f6ec789f8a14aef9ed60cf6681ce254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:11.006037 1572 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Bootstrap starting.
I20260812 03:55:11.006580 1574 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: Bootstrap starting.
I20260812 03:55:11.006816 1572 tablet_bootstrap.cc:654] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:11.007280 1574 tablet_bootstrap.cc:654] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:11.007586 1572 log.cc:824] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:11.007941 1574 log.cc:824] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:11.008302 1573 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Bootstrap starting.
I20260812 03:55:11.008658 1572 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: No bootstrap required, opened a new log
I20260812 03:55:11.008746 1572 ts_tablet_manager.cc:1397] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:55:11.008984 1573 tablet_bootstrap.cc:654] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:11.009017 1574 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: No bootstrap required, opened a new log
I20260812 03:55:11.009078 1574 ts_tablet_manager.cc:1397] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:55:11.009517 1573 log.cc:824] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:11.010214 1573 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: No bootstrap required, opened a new log
I20260812 03:55:11.010298 1573 ts_tablet_manager.cc:1397] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:55:11.010885 1572 raft_consensus.cc:348] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.011070 1572 raft_consensus.cc:374] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:11.011116 1572 raft_consensus.cc:729] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2969f0211eeb4a1e880d11ef509b1113, State: Initialized, Role: FOLLOWER
I20260812 03:55:11.011090 1574 raft_consensus.cc:348] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.011234 1574 raft_consensus.cc:374] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:11.011278 1574 raft_consensus.cc:729] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77b3a8b3c448f39249216fb32bf7b3, State: Initialized, Role: FOLLOWER
I20260812 03:55:11.011291 1572 consensus_queue.cc:260] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.011392 1574 consensus_queue.cc:260] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.011637 1572 ts_tablet_manager.cc:1428] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:55:11.011656 1574 ts_tablet_manager.cc:1428] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:55:11.011770 1247 heartbeater.cc:503] Master 127.17.54.254:34375 was elected leader, sending a full tablet report...
I20260812 03:55:11.011812 1553 heartbeater.cc:503] Master 127.17.54.254:34375 was elected leader, sending a full tablet report...
I20260812 03:55:11.012180 1573 raft_consensus.cc:348] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.012347 1573 raft_consensus.cc:374] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:11.012382 1573 raft_consensus.cc:729] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0150b6327374a9fa5e38591fe5264e1, State: Initialized, Role: FOLLOWER
I20260812 03:55:11.012539 1573 consensus_queue.cc:260] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.012885 1573 ts_tablet_manager.cc:1428] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:55:11.013018 1400 heartbeater.cc:503] Master 127.17.54.254:34375 was elected leader, sending a full tablet report...
I20260812 03:55:11.036046 1582 raft_consensus.cc:482] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:55:11.036235 1582 raft_consensus.cc:504] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.036653 1582 leader_election.cc:288] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0150b6327374a9fa5e38591fe5264e1 (127.17.54.194:41081), 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903)
I20260812 03:55:11.041213 1571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.037005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:37645 (local address 127.17.54.194:41081)
0812 03:55:11.037084 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.037086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.037392 (+ 306us) server_negotiation.cc:407] Connection header received
0812 03:55:11.037424 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.037425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.037438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.037562 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:11.037710 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.038554 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.040044 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.040704 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.040781 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.040792 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.041085 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.041133 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:55:11.041213 1570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.036998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:44175 (local address 127.17.54.193:36903)
0812 03:55:11.037128 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.037130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.037139 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:55:11.037170 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.037171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.037185 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.037252 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:11.037517 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.038444 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.040050 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.040678 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.040781 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.040792 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.041075 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.041132 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260812 03:55:11.041227 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.036886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36903 (local address 127.17.54.195:44175)
0812 03:55:11.036988 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.037003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.037071 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.037270 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.037271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.037273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:11.037358 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.037360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.038551 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.038552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.039872 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.039876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.040826 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.040828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.040952 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.040955 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.041015 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.041096 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":52}
I20260812 03:55:11.041244 1584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.036887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:41081 (local address 127.17.54.195:37645)
0812 03:55:11.037276 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.037288 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.037347 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.037581 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.037582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.037584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:11.037636 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.037636 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.038649 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.038650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.039872 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.039876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.040827 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.040828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.040928 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.040934 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.041014 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.041096 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":39,"thread_start_us":276,"threads_started":1}
I20260812 03:55:11.041754 1348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f6ec789f8a14aef9ed60cf6681ce254" candidate_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0150b6327374a9fa5e38591fe5264e1" is_pre_election: true
I20260812 03:55:11.041757 1192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f6ec789f8a14aef9ed60cf6681ce254" candidate_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2969f0211eeb4a1e880d11ef509b1113" is_pre_election: true
I20260812 03:55:11.041955 1192 raft_consensus.cc:2454] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77b3a8b3c448f39249216fb32bf7b3 in term 0.
I20260812 03:55:11.041955 1348 raft_consensus.cc:2454] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77b3a8b3c448f39249216fb32bf7b3 in term 0.
I20260812 03:55:11.042239 1425 leader_election.cc:302] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2969f0211eeb4a1e880d11ef509b1113, 4d77b3a8b3c448f39249216fb32bf7b3; no voters:
I20260812 03:55:11.042522 1582 raft_consensus.cc:2781] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:55:11.042593 1582 raft_consensus.cc:482] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:55:11.042634 1582 raft_consensus.cc:3037] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:11.043624 1582 raft_consensus.cc:504] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.043820 1582 leader_election.cc:288] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [CANDIDATE]: Term 1 election: Requested vote from peers e0150b6327374a9fa5e38591fe5264e1 (127.17.54.194:41081), 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903)
I20260812 03:55:11.044039 1192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f6ec789f8a14aef9ed60cf6681ce254" candidate_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2969f0211eeb4a1e880d11ef509b1113"
I20260812 03:55:11.044042 1348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f6ec789f8a14aef9ed60cf6681ce254" candidate_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0150b6327374a9fa5e38591fe5264e1"
I20260812 03:55:11.044160 1192 raft_consensus.cc:3037] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:11.044157 1348 raft_consensus.cc:3037] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:11.044973 1348 raft_consensus.cc:2454] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77b3a8b3c448f39249216fb32bf7b3 in term 1.
I20260812 03:55:11.045006 1192 raft_consensus.cc:2454] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77b3a8b3c448f39249216fb32bf7b3 in term 1.
I20260812 03:55:11.045177 1424 leader_election.cc:302] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77b3a8b3c448f39249216fb32bf7b3, e0150b6327374a9fa5e38591fe5264e1; no voters:
I20260812 03:55:11.045298 1582 raft_consensus.cc:2781] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:11.045500 1582 raft_consensus.cc:686] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 LEADER]: Becoming Leader. State: Replica: 4d77b3a8b3c448f39249216fb32bf7b3, State: Running, Role: LEADER
I20260812 03:55:11.045625 1582 consensus_queue.cc:237] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
W20260812 03:55:11.046887 1401 tablet.cc:2369] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:55:11.046792 1018 catalog_manager.cc:5495] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } health_report { overall_health: UNKNOWN } } }
I20260812 03:55:11.060475 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.057859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.17.54.195:39237)
0812 03:55:11.057934 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.057936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.057965 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:55:11.058007 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.058008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.058031 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.058130 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:55:11.058290 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.058818 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.059621 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.059837 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.059914 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.060082 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:55:11.060201 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:55:11.060294 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.060317 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.060341 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:55:11.062574 1348 raft_consensus.cc:1261] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Refusing update from remote peer 4d77b3a8b3c448f39249216fb32bf7b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:11.062574 1192 raft_consensus.cc:1261] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Refusing update from remote peer 4d77b3a8b3c448f39249216fb32bf7b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:11.063015 1582 consensus_queue.cc:1035] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:11.063246 1585 consensus_queue.cc:1035] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:11.066119 1591 mvcc.cc:205] Tried to move back new op lower bound from 7317532307709018112 to 7317532307643936768. Current Snapshot: MvccSnapshot[applied={T|T < 7317532307709018112}]
I20260812 03:55:11.066341 1592 mvcc.cc:205] Tried to move back new op lower bound from 7317532307709018112 to 7317532307643936768. Current Snapshot: MvccSnapshot[applied={T|T < 7317532307709018112}]
I20260812 03:55:11.068136 1020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:11.068248 1020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:11.069065 1020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Sending DeleteTablet for 3 replicas of tablet 5f6ec789f8a14aef9ed60cf6681ce254
I20260812 03:55:11.069440 1170 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f6ec789f8a14aef9ed60cf6681ce254 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:11 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:44442
I20260812 03:55:11.069439 1480 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f6ec789f8a14aef9ed60cf6681ce254 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:11 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:38336
I20260812 03:55:11.069691 1602 tablet_replica.cc:330] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: stopping tablet replica
I20260812 03:55:11.069701 1326 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f6ec789f8a14aef9ed60cf6681ce254 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:11 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:39870
I20260812 03:55:11.069846 1602 raft_consensus.cc:2229] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:55:11.069947 1603 tablet_replica.cc:330] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: stopping tablet replica
I20260812 03:55:11.069967 1602 raft_consensus.cc:2258] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:11.069960 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 1094
I20260812 03:55:11.070084 1603 raft_consensus.cc:2229] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:11.070206 1603 raft_consensus.cc:2258] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:11.070474 1602 ts_tablet_manager.cc:1905] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:11.070742 1603 ts_tablet_manager.cc:1905] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:11.072350 1602 ts_tablet_manager.cc:1918] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:11.072456 1602 log.cc:1196] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/5f6ec789f8a14aef9ed60cf6681ce254
I20260812 03:55:11.072880 1602 ts_tablet_manager.cc:1939] T 5f6ec789f8a14aef9ed60cf6681ce254 P 4d77b3a8b3c448f39249216fb32bf7b3: Deleting consensus metadata
I20260812 03:55:11.073129 1603 ts_tablet_manager.cc:1918] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:11.073257 1603 log.cc:1196] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/5f6ec789f8a14aef9ed60cf6681ce254
I20260812 03:55:11.073412 1003 catalog_manager.cc:4841] TS 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237): tablet 5f6ec789f8a14aef9ed60cf6681ce254 (table test-table [id=40683552b2b2461e86c00e07ccf329b4]) successfully deleted
I20260812 03:55:11.073668 1603 ts_tablet_manager.cc:1939] T 5f6ec789f8a14aef9ed60cf6681ce254 P e0150b6327374a9fa5e38591fe5264e1: Deleting consensus metadata
I20260812 03:55:11.074231 1003 catalog_manager.cc:4841] TS e0150b6327374a9fa5e38591fe5264e1 (127.17.54.194:41081): tablet 5f6ec789f8a14aef9ed60cf6681ce254 (table test-table [id=40683552b2b2461e86c00e07ccf329b4]) successfully deleted
W20260812 03:55:11.087419 1004 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:36903 (error 108)
I20260812 03:55:11.088068 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.087916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36903 (local address 127.0.0.1:44458)
0812 03:55:11.087979 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.088020 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:36903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:55:11.088133 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 1252
W20260812 03:55:11.088214 1004 catalog_manager.cc:4584] TS 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f6ec789f8a14aef9ed60cf6681ce254: Network error: Client connection negotiation failed: client connection to 127.17.54.193:36903: connect: Connection refused (error 111)
I20260812 03:55:11.095839 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 1405
I20260812 03:55:11.104465 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 986
I20260812 03:55:11.113013 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34375
--webserver_interface=127.17.54.254
--webserver_port=33427
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:34375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:11.196992 1605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:11.197203 1605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:11.197227 1605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:11.198469 1605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:11.198513 1605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:11.198527 1605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:11.198540 1605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:11.198554 1605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:11.200081 1605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:34375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=33427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:11.200361 1605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:11.200631 1605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:55:11.210899 1605 server_base.cc:1034] running on GCE node
W20260812 03:55:11.210851 1614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.210819 1613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.210850 1616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:11.211521 1605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:11.211858 1605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:11.213076 1605 hybrid_clock.cc:648] HybridClock initialized: now 1786506911213049 us; error 39 us; skew 500 ppm
Aug 12 03:55:11 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:11.216195 1605 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:55:11.216766 1605 webserver.cc:460] Webserver started at http://127.17.54.254:33427/ using document root <none> and password file <none>
I20260812 03:55:11.217065 1605 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:11.217160 1605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:11.218762 1605 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:55:11.219607 1623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.219915 1605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:11.220057 1605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "f77bb8b5ab924caaa8ea3be6532ba53e"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "ca27a020283f1e00feae064cec4e8a63"
server_key_iv: "ed097a5acf4dfc098b99aa6665db3b86"
server_key_version: "encryptionkey@0"
I20260812 03:55:11.220525 1605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:11.245591 1605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:11.248154 1605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:11.248334 1605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:11.252907 1605 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:34375
I20260812 03:55:11.252949 1683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:34375 every 8 connection(s)
I20260812 03:55:11.253393 1605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:55:11.255491 1684 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:55:11.256258 1684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Bootstrap starting.
I20260812 03:55:11.259310 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 1605
I20260812 03:55:11.259701 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:36903
--local_ip_for_outbound_sockets=127.17.54.193
--tserver_master_addrs=127.17.54.254:34375
--webserver_port=41709
--webserver_interface=127.17.54.193
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:11.267679 1684 log.cc:824] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:11.270797 1684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:11.271188 1684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Bootstrap complete.
I20260812 03:55:11.274950 1684 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:11.275266 1684 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77bb8b5ab924caaa8ea3be6532ba53e, State: Initialized, Role: FOLLOWER
I20260812 03:55:11.275445 1684 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:11.275516 1684 raft_consensus.cc:388] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:55:11.275560 1684 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:55:11.275610 1684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:55:11.276997 1684 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:11.277112 1684 leader_election.cc:302] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f77bb8b5ab924caaa8ea3be6532ba53e; no voters:
I20260812 03:55:11.277303 1684 leader_election.cc:288] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:55:11.277387 1688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:55:11.277557 1684 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:11.277590 1688 raft_consensus.cc:686] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 2 LEADER]: Becoming Leader. State: Replica: f77bb8b5ab924caaa8ea3be6532ba53e, State: Running, Role: LEADER
I20260812 03:55:11.277846 1688 consensus_queue.cc:237] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:11.278182 1689 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } } }
I20260812 03:55:11.278264 1689 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:11.278402 1690 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f77bb8b5ab924caaa8ea3be6532ba53e. Latest consensus state: current_term: 2 leader_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } } }
I20260812 03:55:11.278469 1690 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:11.278890 1699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:55:11.279799 1699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:55:11.279917 1699 catalog_manager.cc:1259] Loaded cluster ID: 1ac9e4b1bb484e2da311d19776059dd4
I20260812 03:55:11.279954 1699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:55:11.280182 1699 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 03:55:11.284828 1699 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 03:55:11.287484 1699 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:55:11.287724 1699 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Loaded TSK: 0
I20260812 03:55:11.287958 1699 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:55:11.358596 1686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:11.358915 1686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:11.358958 1686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:11.360204 1686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:11.360282 1686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:11.360329 1686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:55:11.361915 1686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:36903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=41709
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:11.362200 1686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:11.362497 1686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:11.368006 1717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.368014 1716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.368040 1719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:11.368904 1686 server_base.cc:1034] running on GCE node
I20260812 03:55:11.369135 1686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:11.369424 1686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:11.370626 1686 hybrid_clock.cc:648] HybridClock initialized: now 1786506911370596 us; error 44 us; skew 500 ppm
Aug 12 03:55:11 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:11.373499 1686 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:55:11.374080 1686 webserver.cc:460] Webserver started at http://127.17.54.193:41709/ using document root <none> and password file <none>
I20260812 03:55:11.374380 1686 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:11.374473 1686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:11.376075 1686 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:11.376998 1727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.377321 1686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:11.377442 1686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "2969f0211eeb4a1e880d11ef509b1113"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "1a970f512a298f9ad561d15dd4fcc961"
server_key_iv: "84f1accea85a03dae29451c5cf2bf43d"
server_key_version: "encryptionkey@0"
I20260812 03:55:11.377987 1686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:11.392477 1686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:11.395094 1686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:11.395275 1686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:11.395557 1686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:11.396118 1737 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:55:11.397616 1686 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:55:11.397676 1686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 03:55:11.397706 1686 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:55:11.398504 1686 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:55:11.398555 1686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:11.398595 1737 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Bootstrap starting.
I20260812 03:55:11.407506 1686 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:36903
I20260812 03:55:11.407582 1859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:36903 every 8 connection(s)
I20260812 03:55:11.408159 1686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:55:11.409803 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 1686
I20260812 03:55:11.410151 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:41081
--local_ip_for_outbound_sockets=127.17.54.194
--tserver_master_addrs=127.17.54.254:34375
--webserver_port=42457
--webserver_interface=127.17.54.194
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:11.411247 1737 log.cc:824] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:11.413803 1737 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:11.414127 1737 tablet_bootstrap.cc:492] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Bootstrap complete.
I20260812 03:55:11.415431 1737 ts_tablet_manager.cc:1397] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.015s
I20260812 03:55:11.416723 1737 raft_consensus.cc:348] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.416987 1737 raft_consensus.cc:729] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2969f0211eeb4a1e880d11ef509b1113, State: Initialized, Role: FOLLOWER
I20260812 03:55:11.417220 1737 consensus_queue.cc:260] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.417554 1737 ts_tablet_manager.cc:1428] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
Aug 12 03:55:11 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:11.422147 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.409125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:37377 (local address 127.17.54.254:34375)
0812 03:55:11.409311 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.409320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.409598 (+ 278us) server_negotiation.cc:407] Connection header received
0812 03:55:11.410448 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.410456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.410526 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.410634 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:11.411732 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.413136 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.414739 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.415448 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.415583 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.418447 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:11.418451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:11.418452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:11.418494 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:11.420245 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:11.420731 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:11.420734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:11.420735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:11.420835 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:11.421113 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:11.421114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:11.421114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:11.421353 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:11.421464 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.421811 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.421941 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":79,"threads_started":1}
I20260812 03:55:11.422148 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.409056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:37377)
0812 03:55:11.409409 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.409466 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.410167 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.410820 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.410824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.411327 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:11.411595 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.411602 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.413361 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.413362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.414646 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.414649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.415648 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.415650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.415802 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.416457 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:11.416468 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:11.418337 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:11.420365 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:11.420375 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:11.420378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:11.420636 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:11.420918 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:11.420919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:11.420919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:11.421043 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:11.421498 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:11.421506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:11.421630 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.421741 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.421937 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":119,"threads_started":1}
I20260812 03:55:11.422950 1860 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34375
I20260812 03:55:11.423082 1860 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:11.423276 1860 heartbeater.cc:511] Master 127.17.54.254:34375 requested a full tablet report, sending...
I20260812 03:55:11.424233 1643 ts_manager.cc:194] Registered new tserver with Master: 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903)
I20260812 03:55:11.425583 1643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:37377
I20260812 03:55:11.426577 1860 heartbeater.cc:503] Master 127.17.54.254:34375 was elected leader, sending a full tablet report...
I20260812 03:55:11.430035 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.425276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44468 (local address 127.17.54.193:36903)
0812 03:55:11.425457 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.425461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.426642 (+ 1181us) server_negotiation.cc:407] Connection header received
0812 03:55:11.426680 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.426682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.426698 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.426770 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:11.427003 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.427737 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.429079 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.429595 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.429690 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.429751 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.429910 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.429956 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260812 03:55:11.430047 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.425007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36903 (local address 127.0.0.1:44468)
0812 03:55:11.426441 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.426482 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.426558 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.426843 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.426846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.426849 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:11.426917 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.426919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.427850 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.427851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.428942 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.428946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.429692 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.429693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.429784 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.429807 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.429893 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.429974 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1400,"thread_start_us":105,"threads_started":1}
I20260812 03:55:11.430831 1787 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f6ec789f8a14aef9ed60cf6681ce254 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:11 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:44468
I20260812 03:55:11.431388 1871 tablet_replica.cc:330] stopping tablet replica
I20260812 03:55:11.431520 1871 raft_consensus.cc:2229] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:11.431685 1871 raft_consensus.cc:2258] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:11.432039 1871 ts_tablet_manager.cc:1905] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:11.434233 1871 ts_tablet_manager.cc:1918] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:11.434406 1871 log.cc:1196] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/5f6ec789f8a14aef9ed60cf6681ce254
I20260812 03:55:11.434659 1871 ts_tablet_manager.cc:1939] T 5f6ec789f8a14aef9ed60cf6681ce254 P 2969f0211eeb4a1e880d11ef509b1113: Deleting consensus metadata
I20260812 03:55:11.435249 1625 catalog_manager.cc:4841] TS 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903): tablet 5f6ec789f8a14aef9ed60cf6681ce254 (table test-table [id=40683552b2b2461e86c00e07ccf329b4]) successfully deleted
W20260812 03:55:11.506512 1866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:11.506729 1866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:11.506753 1866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:11.507889 1866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:11.507929 1866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:11.507962 1866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:55:11.509398 1866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:41081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=42457
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:11.509620 1866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:11.509820 1866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:11.515251 1876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.515236 1877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.515241 1879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:11.515468 1866 server_base.cc:1034] running on GCE node
I20260812 03:55:11.515739 1866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:11.516100 1866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:11.517333 1866 hybrid_clock.cc:648] HybridClock initialized: now 1786506911517287 us; error 60 us; skew 500 ppm
Aug 12 03:55:11 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:11.520195 1866 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:55:11.520747 1866 webserver.cc:460] Webserver started at http://127.17.54.194:42457/ using document root <none> and password file <none>
I20260812 03:55:11.521018 1866 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:11.521116 1866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:11.522719 1866 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:11.523459 1887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.523736 1866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:11.523835 1866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "e0150b6327374a9fa5e38591fe5264e1"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "73f9717805581c84cbf32d64c1cca6b6"
server_key_iv: "5a881ad3a1f5e13ebd1b33425f63892b"
server_key_version: "encryptionkey@0"
I20260812 03:55:11.524287 1866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:11.565382 1866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:11.567844 1866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:11.568059 1866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:11.568311 1866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:11.568650 1866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:11.568708 1866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.568750 1866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:11.568796 1866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.574873 1866 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:41081
I20260812 03:55:11.574963 2017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:41081 every 8 connection(s)
I20260812 03:55:11.575320 1866 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:55:11 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:11.584111 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 1866
I20260812 03:55:11.584415 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:39237
--local_ip_for_outbound_sockets=127.17.54.195
--tserver_master_addrs=127.17.54.254:34375
--webserver_port=32967
--webserver_interface=127.17.54.195
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:11.585594 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.576155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:51703 (local address 127.17.54.254:34375)
0812 03:55:11.576253 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.576256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.576472 (+ 216us) server_negotiation.cc:407] Connection header received
0812 03:55:11.576955 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.576956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.576971 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.577023 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:11.578061 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.578778 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.579836 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.580051 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.580112 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.582449 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:11.582451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:11.582451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:11.582458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:11.584077 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:11.584528 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:11.584530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:11.584530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:11.584599 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:11.584853 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:11.584854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:11.584854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:11.585014 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:11.585087 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.585346 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.585450 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:55:11.585636 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.576122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:51703)
0812 03:55:11.576342 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.576383 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.576865 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.577082 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.577086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.577761 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:11.577957 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.577962 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.578864 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.578865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.579715 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.579717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.580162 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.580163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.580281 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.580865 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:11.580873 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:11.582344 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:11.584155 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:11.584162 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:11.584164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:11.584420 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:11.584680 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:11.584681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:11.584681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:11.584787 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:11.585081 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:11.585086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:11.585205 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.585300 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.585428 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":135,"threads_started":1}
I20260812 03:55:11.586199 2018 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34375
I20260812 03:55:11.586345 2018 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:11.586542 2018 heartbeater.cc:511] Master 127.17.54.254:34375 requested a full tablet report, sending...
I20260812 03:55:11.586997 1643 ts_manager.cc:194] Registered new tserver with Master: e0150b6327374a9fa5e38591fe5264e1 (127.17.54.194:41081)
I20260812 03:55:11.587519 1643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:51703
W20260812 03:55:11.670594 2021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:11.670836 2021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:11.670877 2021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:11.672075 2021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:11.672144 2021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:11.672187 2021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:55:11.673712 2021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:39237
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=32967
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:11.674005 2021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:11.674258 2021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:11.679873 2029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.679934 2026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.679977 2027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:11.680359 2021 server_base.cc:1034] running on GCE node
I20260812 03:55:11.680581 2021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:11.680912 2021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:11.682121 2021 hybrid_clock.cc:648] HybridClock initialized: now 1786506911682096 us; error 42 us; skew 500 ppm
Aug 12 03:55:11 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:11.685081 2021 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:55:11.685582 2021 webserver.cc:460] Webserver started at http://127.17.54.195:32967/ using document root <none> and password file <none>
I20260812 03:55:11.685853 2021 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:11.685956 2021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:11.687485 2021 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:11.688230 2037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.688508 2021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:11.688642 2021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "4d77b3a8b3c448f39249216fb32bf7b3"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "4679029687bf61f4ef98df74d697b915"
server_key_iv: "3e79e8f03fb535c3d57371d8817b04da"
server_key_version: "encryptionkey@0"
I20260812 03:55:11.689080 2021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:11.701558 2021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:11.704066 2021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:11.704259 2021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:11.704557 2021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:11.704890 2021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:11.704945 2021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.704993 2021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:11.705034 2021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.711282 2021 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:39237
I20260812 03:55:11.711390 2168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:39237 every 8 connection(s)
I20260812 03:55:11.711743 2021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:55:11 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:11.721414 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 2021
I20260812 03:55:11.722078 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.712554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:38249 (local address 127.17.54.254:34375)
0812 03:55:11.712650 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.712653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.712928 (+ 275us) server_negotiation.cc:407] Connection header received
0812 03:55:11.713406 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.713407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.713418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.713472 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:11.714495 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.715031 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.716160 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.716404 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.716472 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.719037 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:11.719038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:11.719038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:11.719046 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:11.720470 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:11.720930 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:11.720931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:11.720932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:11.721000 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:11.721284 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:11.721285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:11.721286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:11.721464 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:11.721548 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.721890 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.721988 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:55:11.722293 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.712548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:38249)
0812 03:55:11.712791 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.712832 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.713313 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.713525 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.713529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.714106 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:11.714386 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.714393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.715153 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.715154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.716023 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.716027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.716532 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.716533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.716726 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.717287 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:11.717296 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:11.718929 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:11.720566 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:11.720574 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:11.720576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:11.720859 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:11.721089 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:11.721090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:11.721091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:11.721207 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:11.721562 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:11.721569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:11.721720 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.721841 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.722033 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":125,"threads_started":1}
I20260812 03:55:11.722956 2170 heartbeater.cc:347] Connected to a master server at 127.17.54.254:34375
I20260812 03:55:11.723089 2170 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:11.723294 2170 heartbeater.cc:511] Master 127.17.54.254:34375 requested a full tablet report, sending...
I20260812 03:55:11.723757 1643 ts_manager.cc:194] Registered new tserver with Master: 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237)
I20260812 03:55:11.724268 1643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:38249
I20260812 03:55:11.729737 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.721867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.17.54.254:34375)
0812 03:55:11.721989 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.721991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.722265 (+ 274us) server_negotiation.cc:407] Connection header received
0812 03:55:11.722296 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.722297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.722464 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.722555 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:11.723889 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.724644 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.725452 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.725669 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.725748 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.727047 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:11.727049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:11.727049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:11.727057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:11.728522 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:11.728868 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:11.728869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:11.728870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:11.728944 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:11.729169 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:11.729171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:11.729171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:11.729295 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:11.729507 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.729546 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.729628 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260812 03:55:11.730751 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:55:11.733963 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.731151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.17.54.254:34375)
0812 03:55:11.731194 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.731196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.731279 (+ 83us) server_negotiation.cc:407] Connection header received
0812 03:55:11.731327 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.731328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.731359 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.731439 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:55:11.731564 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.732083 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.732922 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.733124 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.733189 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.733398 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:55:11.733697 (+ 299us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:55:11.733787 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.733872 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.733902 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:55:11.734277 1643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:55:11.736064 1643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:55:11.738730 1787 tablet_service.cc:1467] Processing CreateTablet for tablet 1e17f5301b1d457297f0f29b2da1af4f (DEFAULT_TABLE table=test-table [id=084b9e9b9e6c407f82b6badcb9de80fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:11.738987 1787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e17f5301b1d457297f0f29b2da1af4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:11.740679 1737 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Bootstrap starting.
I20260812 03:55:11.741072 1737 tablet_bootstrap.cc:654] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:11.742493 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.738478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:41081 (local address 127.0.0.1:39880)
0812 03:55:11.738701 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.738712 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.738752 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.739008 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.739009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.739011 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:11.739061 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.739063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.740324 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.740325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.741326 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.741354 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.742204 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.742205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.742266 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.742269 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.742366 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.742416 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260812 03:55:11.742509 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.738568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.17.54.194:41081)
0812 03:55:11.738833 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.738838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.738848 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:55:11.738890 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.738893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.738909 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.738992 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:11.739157 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.740196 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.741462 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.742117 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.742206 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.742269 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.742389 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.742431 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":139,"threads_started":1}
I20260812 03:55:11.743368 1943 tablet_service.cc:1467] Processing CreateTablet for tablet 1e17f5301b1d457297f0f29b2da1af4f (DEFAULT_TABLE table=test-table [id=084b9e9b9e6c407f82b6badcb9de80fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:11.743801 1943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e17f5301b1d457297f0f29b2da1af4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:11.744645 1737 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: No bootstrap required, opened a new log
I20260812 03:55:11.744700 1737 ts_tablet_manager.cc:1397] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 03:55:11.744827 1737 raft_consensus.cc:348] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.744899 1737 raft_consensus.cc:374] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:11.745012 1737 raft_consensus.cc:729] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2969f0211eeb4a1e880d11ef509b1113, State: Initialized, Role: FOLLOWER
I20260812 03:55:11.745107 1737 consensus_queue.cc:260] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.745263 1737 ts_tablet_manager.cc:1428] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:11.746618 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.738593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:39237 (local address 127.0.0.1:38358)
0812 03:55:11.738724 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.738737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.738792 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.741652 (+ 2860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.741653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.741656 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:11.741720 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.741721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.743585 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.743586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.744867 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.744871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.746287 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.746289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.746418 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.746423 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.746486 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.746535 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260812 03:55:11.746703 2180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.740834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38358 (local address 127.17.54.195:39237)
0812 03:55:11.741458 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.741464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.741477 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:55:11.741521 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.741523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.741542 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.741642 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:11.741810 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.743436 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.744991 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.746113 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.746187 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.746247 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.746506 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.746557 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":122,"threads_started":1}
I20260812 03:55:11.748031 2092 tablet_service.cc:1467] Processing CreateTablet for tablet 1e17f5301b1d457297f0f29b2da1af4f (DEFAULT_TABLE table=test-table [id=084b9e9b9e6c407f82b6badcb9de80fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:11.748214 2182 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Bootstrap starting.
I20260812 03:55:11.748464 2092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e17f5301b1d457297f0f29b2da1af4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:11.748926 2182 tablet_bootstrap.cc:654] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:11.749436 2182 log.cc:824] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:11.750293 2182 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: No bootstrap required, opened a new log
I20260812 03:55:11.750502 2182 ts_tablet_manager.cc:1397] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:55:11.751688 2184 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Bootstrap starting.
I20260812 03:55:11.752522 2184 tablet_bootstrap.cc:654] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:11.752818 2182 raft_consensus.cc:348] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.753048 2182 raft_consensus.cc:374] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:11.753087 2182 raft_consensus.cc:729] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0150b6327374a9fa5e38591fe5264e1, State: Initialized, Role: FOLLOWER
I20260812 03:55:11.753206 2184 log.cc:824] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:11.753235 2182 consensus_queue.cc:260] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.753626 2182 ts_tablet_manager.cc:1428] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:55:11.753664 2018 heartbeater.cc:503] Master 127.17.54.254:34375 was elected leader, sending a full tablet report...
I20260812 03:55:11.754264 2184 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: No bootstrap required, opened a new log
I20260812 03:55:11.754386 2184 ts_tablet_manager.cc:1397] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260812 03:55:11.756350 2184 raft_consensus.cc:348] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.756513 2184 raft_consensus.cc:374] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:11.756537 2184 raft_consensus.cc:729] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77b3a8b3c448f39249216fb32bf7b3, State: Initialized, Role: FOLLOWER
I20260812 03:55:11.756693 2184 consensus_queue.cc:260] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.757001 2184 ts_tablet_manager.cc:1428] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 03:55:11.757009 2170 heartbeater.cc:503] Master 127.17.54.254:34375 was elected leader, sending a full tablet report...
I20260812 03:55:11.815584 2186 raft_consensus.cc:482] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:55:11.815721 2186 raft_consensus.cc:504] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.816148 2186 leader_election.cc:288] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237), 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903)
I20260812 03:55:11.819281 2180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.816396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:56441 (local address 127.17.54.195:39237)
0812 03:55:11.816475 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.816478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.816490 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:55:11.816563 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.816564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.816572 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.816625 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:11.816846 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.817481 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.818518 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.818915 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.819001 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.819011 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.819170 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.819210 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:55:11.819281 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.816378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:39237 (local address 127.17.54.194:56441)
0812 03:55:11.816438 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.816451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.816501 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.816655 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.816656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.816658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:11.816740 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.816746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.817586 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.817587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.818426 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.818429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.819007 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.819008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.819095 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.819098 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.819163 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.819206 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 03:55:11.819456 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.816454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:53999 (local address 127.17.54.193:36903)
0812 03:55:11.816502 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.816504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.816841 (+ 337us) server_negotiation.cc:407] Connection header received
0812 03:55:11.816952 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.816954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.816966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.817021 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:11.817230 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.817696 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.818709 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.819086 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.819174 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.819183 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.819347 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.819390 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:55:11.819469 2188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.816380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36903 (local address 127.17.54.194:53999)
0812 03:55:11.816763 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.816772 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:11.816853 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:11.817042 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:11.817043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:11.817045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:11.817141 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.817147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.817785 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.817786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.818619 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:11.818622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.819176 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.819177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:11.819252 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.819255 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:55:11.819338 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:55:11.819379 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":186,"thread_start_us":119,"threads_started":1}
I20260812 03:55:11.819738 2115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e17f5301b1d457297f0f29b2da1af4f" candidate_uuid: "e0150b6327374a9fa5e38591fe5264e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" is_pre_election: true
I20260812 03:55:11.819939 2115 raft_consensus.cc:2454] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0150b6327374a9fa5e38591fe5264e1 in term 0.
I20260812 03:55:11.820207 1888 leader_election.cc:302] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77b3a8b3c448f39249216fb32bf7b3, e0150b6327374a9fa5e38591fe5264e1; no voters:
I20260812 03:55:11.820276 1808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e17f5301b1d457297f0f29b2da1af4f" candidate_uuid: "e0150b6327374a9fa5e38591fe5264e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2969f0211eeb4a1e880d11ef509b1113" is_pre_election: true
I20260812 03:55:11.820374 2186 raft_consensus.cc:2781] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:55:11.820451 1808 raft_consensus.cc:2454] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0150b6327374a9fa5e38591fe5264e1 in term 0.
I20260812 03:55:11.820462 2186 raft_consensus.cc:482] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:55:11.820495 2186 raft_consensus.cc:3037] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:11.821390 2186 raft_consensus.cc:504] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.821611 2186 leader_election.cc:288] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [CANDIDATE]: Term 1 election: Requested vote from peers 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237), 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903)
I20260812 03:55:11.821717 1808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e17f5301b1d457297f0f29b2da1af4f" candidate_uuid: "e0150b6327374a9fa5e38591fe5264e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2969f0211eeb4a1e880d11ef509b1113"
I20260812 03:55:11.821790 2115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e17f5301b1d457297f0f29b2da1af4f" candidate_uuid: "e0150b6327374a9fa5e38591fe5264e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77b3a8b3c448f39249216fb32bf7b3"
I20260812 03:55:11.821820 1808 raft_consensus.cc:3037] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:11.821903 2115 raft_consensus.cc:3037] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:11.822697 1808 raft_consensus.cc:2454] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0150b6327374a9fa5e38591fe5264e1 in term 1.
I20260812 03:55:11.822809 2115 raft_consensus.cc:2454] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0150b6327374a9fa5e38591fe5264e1 in term 1.
I20260812 03:55:11.822919 1889 leader_election.cc:302] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2969f0211eeb4a1e880d11ef509b1113, e0150b6327374a9fa5e38591fe5264e1; no voters:
I20260812 03:55:11.823036 2186 raft_consensus.cc:2781] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:11.823195 2186 raft_consensus.cc:686] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 LEADER]: Becoming Leader. State: Replica: e0150b6327374a9fa5e38591fe5264e1, State: Running, Role: LEADER
I20260812 03:55:11.823307 2186 consensus_queue.cc:237] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:11.824249 1643 catalog_manager.cc:5495] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0150b6327374a9fa5e38591fe5264e1 (127.17.54.194). New cstate: current_term: 1 leader_uuid: "e0150b6327374a9fa5e38591fe5264e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } health_report { overall_health: UNKNOWN } } }
W20260812 03:55:11.825893 2019 tablet.cc:2369] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:55:11.844822 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.842060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39888 (local address 127.17.54.194:41081)
0812 03:55:11.842126 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:11.842128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:11.842183 (+ 55us) server_negotiation.cc:407] Connection header received
0812 03:55:11.842267 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:11.842268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:11.842287 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:11.842409 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:55:11.842606 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.843108 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.843955 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:11.844167 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:11.844228 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:11.844437 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:55:11.844548 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:55:11.844616 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:11.844704 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:55:11.844728 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:55:11.847035 2115 raft_consensus.cc:1261] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 FOLLOWER]: Refusing update from remote peer e0150b6327374a9fa5e38591fe5264e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:11.847064 1808 raft_consensus.cc:1261] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Refusing update from remote peer e0150b6327374a9fa5e38591fe5264e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:11.847431 2189 consensus_queue.cc:1035] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:11.847654 2186 consensus_queue.cc:1035] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:11.850643 2192 mvcc.cc:205] Tried to move back new op lower bound from 7317532310922043392 to 7317532310828961792. Current Snapshot: MvccSnapshot[applied={T|T < 7317532310922043392}]
I20260812 03:55:11.851387 2193 mvcc.cc:205] Tried to move back new op lower bound from 7317532310922043392 to 7317532310828961792. Current Snapshot: MvccSnapshot[applied={T|T < 7317532310922043392 or (T in {7317532310922043392})}]
I20260812 03:55:11.852913 1642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:11.853029 1642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:11.853840 1642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Sending DeleteTablet for 3 replicas of tablet 1e17f5301b1d457297f0f29b2da1af4f
I20260812 03:55:11.854185 1787 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e17f5301b1d457297f0f29b2da1af4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:11 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:44468
I20260812 03:55:11.854177 1943 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e17f5301b1d457297f0f29b2da1af4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:11 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:39880
I20260812 03:55:11.854295 1871 tablet_replica.cc:330] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: stopping tablet replica
I20260812 03:55:11.854280 2092 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e17f5301b1d457297f0f29b2da1af4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:11 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:38358
I20260812 03:55:11.854372 1871 raft_consensus.cc:2229] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:11.854409 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 1686
I20260812 03:55:11.854441 1871 raft_consensus.cc:2258] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:11.856815 2205 tablet_replica.cc:330] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: stopping tablet replica
I20260812 03:55:11.856952 2205 raft_consensus.cc:2229] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:11.857069 2205 raft_consensus.cc:2258] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:11.857569 2205 ts_tablet_manager.cc:1905] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:11.859462 2205 ts_tablet_manager.cc:1918] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:11.859592 2205 log.cc:1196] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/1e17f5301b1d457297f0f29b2da1af4f
I20260812 03:55:11.859951 2205 ts_tablet_manager.cc:1939] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Deleting consensus metadata
I20260812 03:55:11.860526 1624 catalog_manager.cc:4841] TS 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237): tablet 1e17f5301b1d457297f0f29b2da1af4f (table test-table [id=084b9e9b9e6c407f82b6badcb9de80fd]) successfully deleted
W20260812 03:55:11.863322 1625 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:36903 (error 108)
I20260812 03:55:11.863600 2204 tablet_replica.cc:330] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: stopping tablet replica
I20260812 03:55:11.863763 2204 raft_consensus.cc:2229] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:55:11.863933 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.863784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36903 (local address 127.0.0.1:44478)
0812 03:55:11.863853 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.863898 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:36903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 03:55:11.863924 2204 raft_consensus.cc:2258] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:11.864169 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 1866
W20260812 03:55:11.864094 1625 catalog_manager.cc:4584] TS 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e17f5301b1d457297f0f29b2da1af4f: Network error: Client connection negotiation failed: client connection to 127.17.54.193:36903: connect: Connection refused (error 111)
I20260812 03:55:11.871685 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 2021
I20260812 03:55:11.871862 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:11.871695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:41081 (local address 127.0.0.1:39902)
0812 03:55:11.871762 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:11.871797 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.194:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:55:11.879534 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 1605
I20260812 03:55:11.889171 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34375
--webserver_interface=127.17.54.254
--webserver_port=33427
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:34375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:11.974965 2207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:11.975227 2207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:11.975265 2207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:11.976491 2207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:11.976567 2207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:11.976589 2207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:11.976610 2207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:11.976640 2207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:11.978155 2207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:34375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:34375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=33427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:11.978474 2207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:11.978727 2207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:11.983832 2213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.983847 2212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:11.983781 2215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:11.984041 2207 server_base.cc:1034] running on GCE node
I20260812 03:55:11.984257 2207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:11.984591 2207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:11.985737 2207 hybrid_clock.cc:648] HybridClock initialized: now 1786506911985723 us; error 34 us; skew 500 ppm
Aug 12 03:55:11 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:11.988415 2207 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:55:11.988921 2207 webserver.cc:460] Webserver started at http://127.17.54.254:33427/ using document root <none> and password file <none>
I20260812 03:55:11.989198 2207 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:11.989272 2207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:11.990720 2207 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:55:11.991482 2222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:11.991673 2207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:11.991781 2207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "f77bb8b5ab924caaa8ea3be6532ba53e"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "ca27a020283f1e00feae064cec4e8a63"
server_key_iv: "ed097a5acf4dfc098b99aa6665db3b86"
server_key_version: "encryptionkey@0"
I20260812 03:55:11.992163 2207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:12.005106 2207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:12.007385 2207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:12.007572 2207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:12.012069 2207 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:34375
I20260812 03:55:12.012130 2280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:34375 every 8 connection(s)
I20260812 03:55:12.012528 2207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:55:12.014175 2281 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:55:12.014889 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 2207
I20260812 03:55:12.014910 2281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Bootstrap starting.
I20260812 03:55:12.015180 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:36903
--local_ip_for_outbound_sockets=127.17.54.193
--tserver_master_addrs=127.17.54.254:34375
--webserver_port=41709
--webserver_interface=127.17.54.193
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:12.024708 2281 log.cc:824] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:12.027156 2281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:12.027508 2281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e: Bootstrap complete.
I20260812 03:55:12.029990 2281 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:12.030217 2281 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77bb8b5ab924caaa8ea3be6532ba53e, State: Initialized, Role: FOLLOWER
I20260812 03:55:12.030423 2281 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:12.030524 2281 raft_consensus.cc:388] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:55:12.030586 2281 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:55:12.030642 2281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:55:12.031718 2281 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:12.031848 2281 leader_election.cc:302] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f77bb8b5ab924caaa8ea3be6532ba53e; no voters:
I20260812 03:55:12.032078 2281 leader_election.cc:288] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:55:12.032124 2286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:55:12.032250 2286 raft_consensus.cc:686] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [term 3 LEADER]: Becoming Leader. State: Replica: f77bb8b5ab924caaa8ea3be6532ba53e, State: Running, Role: LEADER
I20260812 03:55:12.032343 2286 consensus_queue.cc:237] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } }
I20260812 03:55:12.032390 2281 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:12.032622 2287 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } } }
I20260812 03:55:12.032696 2287 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:12.032842 2287 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f77bb8b5ab924caaa8ea3be6532ba53e. Latest consensus state: current_term: 3 leader_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77bb8b5ab924caaa8ea3be6532ba53e" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 34375 } } }
I20260812 03:55:12.032917 2287 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77bb8b5ab924caaa8ea3be6532ba53e [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:12.033205 2290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:55:12.034121 2290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:55:12.034286 2290 catalog_manager.cc:1259] Loaded cluster ID: 1ac9e4b1bb484e2da311d19776059dd4
I20260812 03:55:12.034597 2290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 03:55:12.034961 2290 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f15b89179ad google::LogMessage::Fail() at ??:0
@ 0x7f15b891a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f15b89174de google::LogMessage::Flush() at ??:0
@ 0x7f15b8918a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f15c08380f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f15c08418b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f15ba7e50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f15ba7da954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f15bb63e6db start_thread at ??:0
@ 0x7f15b741561f clone at ??:0
W20260812 03:55:12.116122 2283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:12.116335 2283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:12.116359 2283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:12.117607 2283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:12.117659 2283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:12.117704 2283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:55:12.119324 2283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:36903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=41709
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:12.119629 2283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:12.119871 2283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:12.125370 2327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:12.125356 2324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:12.125416 2325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:12.125550 2283 server_base.cc:1034] running on GCE node
I20260812 03:55:12.125902 2283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:12.126188 2283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:12.127386 2283 hybrid_clock.cc:648] HybridClock initialized: now 1786506912127379 us; error 45 us; skew 500 ppm
Aug 12 03:55:12 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:12.130165 2283 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:55:12.130728 2283 webserver.cc:460] Webserver started at http://127.17.54.193:41709/ using document root <none> and password file <none>
I20260812 03:55:12.130982 2283 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:12.131059 2283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:12.132550 2283 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:12.133389 2336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:12.133594 2283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:12.133704 2283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "2969f0211eeb4a1e880d11ef509b1113"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "1a970f512a298f9ad561d15dd4fcc961"
server_key_iv: "84f1accea85a03dae29451c5cf2bf43d"
server_key_version: "encryptionkey@0"
I20260812 03:55:12.134093 2283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:12.151099 2283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:12.153602 2283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:12.153781 2283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:12.154052 2283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:12.154615 2343 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:55:12.156134 2283 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:55:12.156190 2283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 03:55:12.156219 2283 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:55:12.156935 2283 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:55:12.156980 2283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:12.157029 2343 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Bootstrap starting.
I20260812 03:55:12.165158 2283 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:36903
I20260812 03:55:12.165383 2457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:36903 every 8 connection(s)
I20260812 03:55:12.165762 2283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:55:12.168891 2343 log.cc:824] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:12.171188 2343 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:12.171489 2343 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Bootstrap complete.
I20260812 03:55:12.171761 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 2283
I20260812 03:55:12.172075 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:41081
--local_ip_for_outbound_sockets=127.17.54.194
--tserver_master_addrs=127.17.54.254:34375
--webserver_port=42457
--webserver_interface=127.17.54.194
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:12.172776 2343 ts_tablet_manager.cc:1397] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.012s
I20260812 03:55:12.174429 2343 raft_consensus.cc:348] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:12.174698 2343 raft_consensus.cc:729] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2969f0211eeb4a1e880d11ef509b1113, State: Initialized, Role: FOLLOWER
I20260812 03:55:12.174876 2343 consensus_queue.cc:260] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:12.175199 2343 ts_tablet_manager.cc:1428] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260812 03:55:12.226526 2462 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:55:12.166535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:36201)
0812 03:55:12.166727 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.166780 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:12.167482 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:12.226258 (+ 58776us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
W20260812 03:55:12.226761 2459 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:34375 (0 consecutive failures): Network error: Failed to ping master at 127.17.54.254:34375: Client connection negotiation failed: client connection to 127.17.54.254:34375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:55:12.227121 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.226987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:55359)
0812 03:55:12.227046 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.227089 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 03:55:12.227628 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.227441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:56347)
0812 03:55:12.227557 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.227596 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 03:55:12.227802 2459 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:55:12.259104 2464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:12.259310 2464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:12.259335 2464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:12.260538 2464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:12.260581 2464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:12.260608 2464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:55:12.262116 2464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:41081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=42457
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:12.262377 2464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:12.262621 2464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:12.268154 2472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:12.268281 2473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:12.268457 2464 server_base.cc:1034] running on GCE node
W20260812 03:55:12.268465 2475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:12.268746 2464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:12.269040 2464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:12.270222 2464 hybrid_clock.cc:648] HybridClock initialized: now 1786506912270219 us; error 41 us; skew 500 ppm
Aug 12 03:55:12 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:12.272965 2464 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:55:12.273485 2464 webserver.cc:460] Webserver started at http://127.17.54.194:42457/ using document root <none> and password file <none>
I20260812 03:55:12.273756 2464 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:12.273837 2464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:12.275286 2464 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:12.276031 2484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:12.276208 2464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:12.276304 2464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "e0150b6327374a9fa5e38591fe5264e1"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "73f9717805581c84cbf32d64c1cca6b6"
server_key_iv: "5a881ad3a1f5e13ebd1b33425f63892b"
server_key_version: "encryptionkey@0"
I20260812 03:55:12.276679 2464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:12.289871 2464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:12.292130 2464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:12.292312 2464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:12.292577 2464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:12.293038 2491 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:55:12.294605 2464 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:55:12.294669 2464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 03:55:12.294732 2464 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:55:12.295498 2464 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:55:12.295571 2464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:12.295666 2491 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Bootstrap starting.
I20260812 03:55:12.304796 2464 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:41081
I20260812 03:55:12.304996 2608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:41081 every 8 connection(s)
I20260812 03:55:12.305320 2464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:55:12.306676 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.306079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:43975)
0812 03:55:12.306281 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.306426 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
W20260812 03:55:12.307022 2609 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:34375 (0 consecutive failures): Network error: Failed to ping master at 127.17.54.254:34375: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
I20260812 03:55:12.307394 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.307256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:36339)
0812 03:55:12.307324 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.307366 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:55:12.307770 2491 log.cc:824] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:12.307847 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.307750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:48393)
0812 03:55:12.307785 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.307823 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 03:55:12.308035 2609 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:55:12.309046 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 2464
I20260812 03:55:12.309391 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:39237
--local_ip_for_outbound_sockets=127.17.54.195
--tserver_master_addrs=127.17.54.254:34375
--webserver_port=32967
--webserver_interface=127.17.54.195
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:39731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:12.310778 2491 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:12.311095 2491 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Bootstrap complete.
I20260812 03:55:12.312263 2491 ts_tablet_manager.cc:1397] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260812 03:55:12.313685 2491 raft_consensus.cc:348] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:12.313995 2491 raft_consensus.cc:729] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0150b6327374a9fa5e38591fe5264e1, State: Initialized, Role: FOLLOWER
I20260812 03:55:12.314208 2491 consensus_queue.cc:260] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:12.314529 2491 ts_tablet_manager.cc:1428] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260812 03:55:12.314954 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.314795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:59869)
0812 03:55:12.314879 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.314927 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 03:55:12.315312 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.315220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:55413)
0812 03:55:12.315267 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.315293 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:55:12.396365 2614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:12.396582 2614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:12.396610 2614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:12.397725 2614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:12.397763 2614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:12.397789 2614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:55:12.399250 2614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:39237
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=32967
--tserver_master_addrs=127.17.54.254:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:12.399514 2614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:12.399709 2614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:12.405249 2621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:12.405249 2620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:12.405329 2623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:12.405637 2614 server_base.cc:1034] running on GCE node
I20260812 03:55:12.405889 2614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:12.406170 2614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:12.407409 2614 hybrid_clock.cc:648] HybridClock initialized: now 1786506912407393 us; error 43 us; skew 500 ppm
Aug 12 03:55:12 dist-test-slave-tm1g krb5kdc[971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:12.410260 2614 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:55:12.410881 2614 webserver.cc:460] Webserver started at http://127.17.54.195:32967/ using document root <none> and password file <none>
I20260812 03:55:12.411166 2614 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:12.411245 2614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:12.412835 2614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:55:12.413618 2631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:12.413833 2614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:12.413960 2614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "4d77b3a8b3c448f39249216fb32bf7b3"
format_stamp: "Formatted at 2026-08-12 03:55:10 on dist-test-slave-tm1g"
server_key: "4679029687bf61f4ef98df74d697b915"
server_key_iv: "3e79e8f03fb535c3d57371d8817b04da"
server_key_version: "encryptionkey@0"
I20260812 03:55:12.414356 2614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:12.434975 2614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:12.437752 2614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:12.437928 2614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:12.438194 2614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:12.438545 2614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:12.438602 2614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:12.438645 2614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:12.438691 2614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:12.444643 2614 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:39237
I20260812 03:55:12.444749 2754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:39237 every 8 connection(s)
I20260812 03:55:12.445101 2614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:55:12.445926 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 2614
I20260812 03:55:12.446691 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.445868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:51471)
0812 03:55:12.446082 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.446190 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
W20260812 03:55:12.446990 2755 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:34375 (0 consecutive failures): Network error: Failed to ping master at 127.17.54.254:34375: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
I20260812 03:55:12.447319 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.447218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:38429)
0812 03:55:12.447272 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.447299 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 03:55:12.447672 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:12.447594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:47693)
0812 03:55:12.447635 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:12.447657 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:55:12.447781 2755 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:55:13.228529 2760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:13.228244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:50089)
0812 03:55:13.228384 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:13.228429 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260812 03:55:13.316105 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:13.315816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:52857)
0812 03:55:13.315989 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:13.316038 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260812 03:55:13.448453 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:13.448177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:41539)
0812 03:55:13.448354 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:13.448396 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
I20260812 03:55:13.822026 2764 raft_consensus.cc:482] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:55:13.822209 2764 raft_consensus.cc:504] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:13.822672 2764 leader_election.cc:288] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e0150b6327374a9fa5e38591fe5264e1 (127.17.54.194:41081), 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237)
Aug 12 03:55:13 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.195@KRBTEST.COM
Aug 12 03:55:13 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.194@KRBTEST.COM
I20260812 03:55:13.834637 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:13.823120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:39237 (local address 127.17.54.193:57155)
0812 03:55:13.823258 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:13.823267 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:13.823314 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:13.824384 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:13.824385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:13.824597 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:13.824647 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:13.824650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:13.825838 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:13.825839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:13.826908 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:13.826911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:13.827612 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:13.827613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:13.827658 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:13.828290 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:13.828309 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:13.830047 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:13.832530 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:13.832538 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:13.832541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:13.832882 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:13.833240 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:13.833241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:13.833241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:13.833424 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:13.833978 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:13.833985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:13.834169 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 03:55:13.834269 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:55:13.834522 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":53,"threads_started":1}
I20260812 03:55:13.834714 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:13.823059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:35519 (local address 127.17.54.194:41081)
0812 03:55:13.823252 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:13.823265 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:13.823293 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:55:13.823437 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:13.823440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:13.823494 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:13.823617 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:13.824551 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:13.825775 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:13.826866 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:13.827355 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:13.827439 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:13.830654 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:13.830658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:13.830658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:13.830669 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:13.832613 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:13.833151 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:13.833154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:13.833154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:13.833350 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:13.833712 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:13.833713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:13.833713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:13.834060 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:13.834159 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:13.834455 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 03:55:13.834593 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":60,"threads_started":1}
I20260812 03:55:13.834743 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:13.823265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:57155 (local address 127.17.54.195:39237)
0812 03:55:13.823475 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:13.823483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:13.823512 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:55:13.824174 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:13.824177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:13.824225 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:13.824368 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:13.824757 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:13.825648 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:13.827032 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:13.827520 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:13.827654 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:13.830186 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:13.830190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:13.830190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:13.830232 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:13.832400 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:13.832983 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:13.832988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:13.832988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:13.833125 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:13.833537 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:13.833539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:13.833539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:13.833874 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:13.833943 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:13.834441 (+ 498us) server_negotiation.cc:299] Negotiation successful
0812 03:55:13.834613 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":125,"threads_started":1}
I20260812 03:55:13.834779 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:13.822893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:41081 (local address 127.17.54.193:35519)
0812 03:55:13.823064 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:13.823075 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:13.823167 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:13.823683 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:13.823686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:13.824108 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:13.824415 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:13.824424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:13.825889 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:13.825890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:13.826756 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:13.826760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:13.827518 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:13.827520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:13.827658 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:13.828348 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:13.828353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:13.830520 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:13.832736 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:13.832737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:13.832737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:13.833052 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:13.833526 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:13.833528 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:13.833528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:13.833608 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:13.834183 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:13.834185 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:13.834225 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 03:55:13.834382 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 03:55:13.834615 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":171,"thread_start_us":94,"threads_started":1}
I20260812 03:55:13.835911 2556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e17f5301b1d457297f0f29b2da1af4f" candidate_uuid: "2969f0211eeb4a1e880d11ef509b1113" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e0150b6327374a9fa5e38591fe5264e1" is_pre_election: true
I20260812 03:55:13.836050 2556 raft_consensus.cc:2454] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2969f0211eeb4a1e880d11ef509b1113 in term 1.
W20260812 03:55:13.836403 2337 leader_election.cc:341] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237): Not found: Tablet not found: 1e17f5301b1d457297f0f29b2da1af4f
I20260812 03:55:13.836483 2337 leader_election.cc:302] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2969f0211eeb4a1e880d11ef509b1113, e0150b6327374a9fa5e38591fe5264e1; no voters: 4d77b3a8b3c448f39249216fb32bf7b3
I20260812 03:55:13.836627 2764 raft_consensus.cc:2781] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:55:13.836712 2764 raft_consensus.cc:482] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:55:13.836766 2764 raft_consensus.cc:3037] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:55:13.837976 2764 raft_consensus.cc:504] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:13.838148 2764 leader_election.cc:288] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [CANDIDATE]: Term 2 election: Requested vote from peers e0150b6327374a9fa5e38591fe5264e1 (127.17.54.194:41081), 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237)
I20260812 03:55:13.838409 2556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e17f5301b1d457297f0f29b2da1af4f" candidate_uuid: "2969f0211eeb4a1e880d11ef509b1113" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e0150b6327374a9fa5e38591fe5264e1"
I20260812 03:55:13.838497 2556 raft_consensus.cc:3037] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 1 FOLLOWER]: Advancing to term 2
W20260812 03:55:13.838593 2337 leader_election.cc:341] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237): Not found: Tablet not found: 1e17f5301b1d457297f0f29b2da1af4f
I20260812 03:55:13.839761 2556 raft_consensus.cc:2454] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2969f0211eeb4a1e880d11ef509b1113 in term 2.
I20260812 03:55:13.839972 2337 leader_election.cc:302] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2969f0211eeb4a1e880d11ef509b1113, e0150b6327374a9fa5e38591fe5264e1; no voters: 4d77b3a8b3c448f39249216fb32bf7b3
I20260812 03:55:13.840085 2764 raft_consensus.cc:2781] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:55:13.840209 2764 raft_consensus.cc:686] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [term 2 LEADER]: Becoming Leader. State: Replica: 2969f0211eeb4a1e880d11ef509b1113, State: Running, Role: LEADER
I20260812 03:55:13.840314 2764 consensus_queue.cc:237] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:13.840587 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:13.840478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:55091)
0812 03:55:13.840526 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:13.840555 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 03:55:14.217368 2556 raft_consensus.cc:1261] T 1e17f5301b1d457297f0f29b2da1af4f P e0150b6327374a9fa5e38591fe5264e1 [term 2 FOLLOWER]: Refusing update from remote peer 2969f0211eeb4a1e880d11ef509b1113: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 03:55:14.217914 2764 consensus_queue.cc:1035] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 03:55:14.219388 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:14.219201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:58887)
0812 03:55:14.219307 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:14.219349 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
I20260812 03:55:14.220373 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:14.220185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:60857)
0812 03:55:14.220300 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:14.220334 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 03:55:14.220535 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:14.220418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:37575)
0812 03:55:14.220474 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:14.220503 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:55:14.220599 2337 consensus_peers.cc:487] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113 -> Peer 4d77b3a8b3c448f39249216fb32bf7b3 (127.17.54.195:39237): Couldn't send request to peer 4d77b3a8b3c448f39249216fb32bf7b3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1e17f5301b1d457297f0f29b2da1af4f. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:55:14.449146 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:14.448908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:50413)
0812 03:55:14.449060 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:14.449098 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260812 03:55:14.626950 2781 ts_tablet_manager.cc:927] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Initiating tablet copy from peer 2969f0211eeb4a1e880d11ef509b1113 (127.17.54.193:36903)
I20260812 03:55:14.627265 2781 tablet_copy_client.cc:316] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: tablet copy: Beginning tablet copy session from remote peer at address 127.17.54.193:36903
Aug 12 03:55:14 dist-test-slave-tm1g krb5kdc[971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:55:14.635998 2783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:14.627713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:57595 (local address 127.17.54.193:36903)
0812 03:55:14.627876 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:14.627881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:14.627891 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:55:14.627917 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:14.627919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:14.627931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:14.628004 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:14.628484 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:14.629276 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:14.630102 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:14.630515 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:14.630580 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:14.632740 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:14.632749 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:14.632750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:14.632757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:14.634724 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:14.635138 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:14.635141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:14.635141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:14.635204 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:14.635445 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:14.635446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:14.635446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:14.635603 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:14.635675 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:14.635857 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 03:55:14.635928 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260812 03:55:14.636003 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:14.627578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36903 (local address 127.17.54.195:57595)
0812 03:55:14.627654 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:14.627677 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:14.627749 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:14.628024 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:14.628031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:14.628313 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:14.628377 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:14.628379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:14.629358 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:14.629359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:14.630012 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:14.630014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:14.630589 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:14.630590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:14.630655 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:14.631190 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:14.631195 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:14.632650 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:14.634811 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:14.634819 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:14.634821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:14.635049 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:14.635272 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:14.635273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:14.635273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:14.635384 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:14.635672 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:14.635680 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:14.635732 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:55:14.635806 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:55:14.635908 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 03:55:14.636549 2430 tablet_copy_service.cc:140] P 2969f0211eeb4a1e880d11ef509b1113: Received BeginTabletCopySession request for tablet 1e17f5301b1d457297f0f29b2da1af4f from peer 4d77b3a8b3c448f39249216fb32bf7b3 ({username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:57595)
I20260812 03:55:14.636650 2430 tablet_copy_service.cc:161] P 2969f0211eeb4a1e880d11ef509b1113: Beginning new tablet copy session on tablet 1e17f5301b1d457297f0f29b2da1af4f from peer 4d77b3a8b3c448f39249216fb32bf7b3 at {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:57595: session id = 4d77b3a8b3c448f39249216fb32bf7b3-1e17f5301b1d457297f0f29b2da1af4f
I20260812 03:55:14.637148 2430 tablet_copy_source_session.cc:205] T 1e17f5301b1d457297f0f29b2da1af4f P 2969f0211eeb4a1e880d11ef509b1113: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:55:14.638142 2781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e17f5301b1d457297f0f29b2da1af4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:14.641016 2781 tablet_copy_client.cc:791] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: tablet copy: Starting download of 0 data blocks...
I20260812 03:55:14.641191 2781 tablet_copy_client.cc:655] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: tablet copy: Starting download of 1 WAL segments...
I20260812 03:55:14.642098 2781 tablet_copy_client.cc:531] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:55:14.642910 2781 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Bootstrap starting.
I20260812 03:55:14.644397 2781 log.cc:824] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:14.647223 2781 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:14.647480 2781 tablet_bootstrap.cc:492] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Bootstrap complete.
I20260812 03:55:14.647635 2781 ts_tablet_manager.cc:1397] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260812 03:55:14.648622 2781 raft_consensus.cc:348] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:14.648825 2781 raft_consensus.cc:729] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77b3a8b3c448f39249216fb32bf7b3, State: Initialized, Role: FOLLOWER
I20260812 03:55:14.649024 2781 consensus_queue.cc:260] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0150b6327374a9fa5e38591fe5264e1" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 41081 } } peers { permanent_uuid: "4d77b3a8b3c448f39249216fb32bf7b3" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 39237 } } peers { permanent_uuid: "2969f0211eeb4a1e880d11ef509b1113" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36903 } }
I20260812 03:55:14.649350 2781 ts_tablet_manager.cc:1428] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 03:55:14.650003 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:14.649818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:46347)
0812 03:55:14.649878 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:14.649968 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 03:55:14.650027 2430 tablet_copy_service.cc:342] P 2969f0211eeb4a1e880d11ef509b1113: Request end of tablet copy session 4d77b3a8b3c448f39249216fb32bf7b3-1e17f5301b1d457297f0f29b2da1af4f received from {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:57595
I20260812 03:55:14.650128 2430 tablet_copy_service.cc:433] P 2969f0211eeb4a1e880d11ef509b1113: ending tablet copy session 4d77b3a8b3c448f39249216fb32bf7b3-1e17f5301b1d457297f0f29b2da1af4f on tablet 1e17f5301b1d457297f0f29b2da1af4f with peer 4d77b3a8b3c448f39249216fb32bf7b3
I20260812 03:55:14.650667 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:14.650553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:57053)
0812 03:55:14.650604 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:14.650635 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:55:15.059580 2704 raft_consensus.cc:1203] T 1e17f5301b1d457297f0f29b2da1af4f P 4d77b3a8b3c448f39249216fb32bf7b3 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260812 03:55:15.059639 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:15.059446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:54551)
0812 03:55:15.059552 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:15.059594 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
I20260812 03:55:15.060073 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:15.059969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:46851)
0812 03:55:15.060005 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:15.060036 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 03:55:15.221247 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:15.220997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:41193)
0812 03:55:15.221149 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:15.221197 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260812 03:55:16.060465 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:16.060176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:50283)
0812 03:55:16.060360 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:16.060412 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260812 03:55:16.060683 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:16.060449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:60647)
0812 03:55:16.060566 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:16.060629 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260812 03:55:16.222107 2791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:16.221842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:40437)
0812 03:55:16.222009 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:16.222055 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260812 03:55:17.061378 2792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:17.061090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:42399)
0812 03:55:17.061261 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:17.061313 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260812 03:55:17.061439 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:17.061207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:44689)
0812 03:55:17.061337 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:17.061381 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260812 03:55:17.222935 2794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:17.222676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:41815)
0812 03:55:17.222844 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:17.222885 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
I20260812 03:55:18.062382 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:18.062034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:39069)
0812 03:55:18.062214 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:18.062273 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260812 03:55:18.062407 2796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:18.062121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:48699)
0812 03:55:18.062252 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:18.062290 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260812 03:55:18.062849 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:18.062690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:34065)
0812 03:55:18.062767 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:18.062807 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 03:55:18.223734 2797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:18.223513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:47525)
0812 03:55:18.223644 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:18.223684 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260812 03:55:19.063166 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:19.062921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:53029)
0812 03:55:19.063072 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:19.063114 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260812 03:55:19.063596 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:19.063348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:36671)
0812 03:55:19.063515 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:19.063555 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260812 03:55:19.224490 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:19.224237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:53127)
0812 03:55:19.224396 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:19.224436 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260812 03:55:20.064128 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:20.063843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:34511)
0812 03:55:20.064032 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:20.064076 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260812 03:55:20.064285 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:20.064050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:53231)
0812 03:55:20.064187 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:20.064232 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260812 03:55:20.225447 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:20.225114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:44559)
0812 03:55:20.225305 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:20.225354 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260812 03:55:21.064935 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:21.064675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:54607)
0812 03:55:21.064842 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:21.064880 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260812 03:55:21.065084 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:21.064804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:53013)
0812 03:55:21.064968 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:21.065027 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260812 03:55:21.226432 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:21.226084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:53735)
0812 03:55:21.226254 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:21.226380 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260812 03:55:22.065881 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:22.065601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:45789)
0812 03:55:22.065790 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:22.065829 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
I20260812 03:55:22.065907 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:22.065601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:54911)
0812 03:55:22.065787 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:22.065836 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
I20260812 03:55:22.227344 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:22.227056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:45827)
0812 03:55:22.227232 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:22.227281 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260812 03:55:23.066886 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:23.066605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:47557)
0812 03:55:23.066794 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:23.066842 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
I20260812 03:55:23.066886 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:23.066582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:57869)
0812 03:55:23.066756 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:23.066808 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260812 03:55:23.228193 2813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:23.227931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:54789)
0812 03:55:23.228092 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:23.228143 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260812 03:55:24.067871 2814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:24.067582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:51497)
0812 03:55:24.067750 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:24.067801 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260812 03:55:24.067965 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:24.067711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:50737)
0812 03:55:24.067854 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:24.067898 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260812 03:55:24.229014 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:24.228757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:41873)
0812 03:55:24.228920 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:24.228965 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260812 03:55:25.068723 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:25.068486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:59823)
0812 03:55:25.068634 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:25.068678 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260812 03:55:25.068723 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:25.068472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:49535)
0812 03:55:25.068634 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:25.068668 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260812 03:55:25.229904 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:25.229614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:52669)
0812 03:55:25.229812 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:25.229852 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
I20260812 03:55:26.069693 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:26.069371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:57277)
0812 03:55:26.069592 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:26.069634 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":109,"threads_started":1}
I20260812 03:55:26.069694 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:26.069371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:43835)
0812 03:55:26.069569 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:26.069623 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":65,"threads_started":1}
I20260812 03:55:26.230758 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:26.230516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:35437)
0812 03:55:26.230669 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:26.230706 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260812 03:55:27.070571 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:27.070223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.195:37721)
0812 03:55:27.070464 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:27.070509 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":61,"threads_started":1}
I20260812 03:55:27.070680 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:27.070256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.193:39385)
0812 03:55:27.070559 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:27.070604 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":245,"threads_started":1}
I20260812 03:55:27.231563 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:27.231280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:34375 (local address 127.17.54.194:41735)
0812 03:55:27.231437 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:27.231510 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:34375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260812 03:55:27.451196 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 2283
I20260812 03:55:27.460795 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 2464
I20260812 03:55:27.469221 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 2614
2026-08-12T03:55:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18467 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:55:27 dist-test-slave-tm1g krb5kdc[2831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:55:27 dist-test-slave-tm1g krb5kdc[2831](info): set up 2 sockets
Aug 12 03:55:27 dist-test-slave-tm1g krb5kdc[2831](info): commencing operation
krb5kdc: starting...
W20260812 03:55:28.612353 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:55:28 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:55:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:55:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:55:28.743360 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33161
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:33161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:28.828408 2846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:28.828671 2846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:28.828711 2846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:28.829918 2846 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:55:28.829990 2846 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:55:28.830014 2846 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:55:28.830034 2846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:28.830063 2846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:28.830096 2846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:28.830119 2846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:28.830145 2846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:28.831702 2846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:33161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:28.832010 2846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:28.832254 2846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:28.837931 2852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:28.837961 2846 server_base.cc:1034] running on GCE node
W20260812 03:55:28.837931 2854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:28.837944 2856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:28.838498 2846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:28.838795 2846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:28.839999 2846 hybrid_clock.cc:648] HybridClock initialized: now 1786506928839982 us; error 30 us; skew 500 ppm
Aug 12 03:55:28 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506928, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:28.842687 2846 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:55:28.843142 2846 webserver.cc:460] Webserver started at http://127.17.54.254:37679/ using document root <none> and password file <none>
I20260812 03:55:28.843377 2846 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:28.843453 2846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:28.843611 2846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:28.844758 2846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "6aedbf489d004bf39c6c44864d17dfe0"
format_stamp: "Formatted at 2026-08-12 03:55:28 on dist-test-slave-tm1g"
server_key: "4b35b77a209f9b3f3833444b266b3d2b"
server_key_iv: "93de43ebf4edbf7c293beff6123d72b9"
server_key_version: "encryptionkey@0"
I20260812 03:55:28.845135 2846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "6aedbf489d004bf39c6c44864d17dfe0"
format_stamp: "Formatted at 2026-08-12 03:55:28 on dist-test-slave-tm1g"
server_key: "4b35b77a209f9b3f3833444b266b3d2b"
server_key_iv: "93de43ebf4edbf7c293beff6123d72b9"
server_key_version: "encryptionkey@0"
I20260812 03:55:28.846625 2846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 03:55:28.847430 2867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:28.847635 2846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:28.847747 2846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "6aedbf489d004bf39c6c44864d17dfe0"
format_stamp: "Formatted at 2026-08-12 03:55:28 on dist-test-slave-tm1g"
server_key: "4b35b77a209f9b3f3833444b266b3d2b"
server_key_iv: "93de43ebf4edbf7c293beff6123d72b9"
server_key_version: "encryptionkey@0"
I20260812 03:55:28.847824 2846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:28.855513 2846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:28.857511 2846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:28.857681 2846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:28.862167 2846 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:33161
I20260812 03:55:28.862201 2923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:33161 every 8 connection(s)
I20260812 03:55:28.862634 2846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:55:28.863531 2924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:28.865989 2924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Bootstrap starting.
I20260812 03:55:28.866714 2924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:28.867036 2924 log.cc:824] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:28.867781 2924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: No bootstrap required, opened a new log
I20260812 03:55:28.869313 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 2846
I20260812 03:55:28.869441 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:55:28.869652 17627 external_mini_cluster.cc:1428] Setting key 611f9d500ab5b11512196e610c411701
I20260812 03:55:28.869756 2924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:55:28.869933 2924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:28.869966 2924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aedbf489d004bf39c6c44864d17dfe0, State: Initialized, Role: FOLLOWER
I20260812 03:55:28.870420 2924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:55:28.870548 2924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:55:28.870576 2924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:55:28.870652 2924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:28.871870 2924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:55:28.871963 2924 leader_election.cc:302] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aedbf489d004bf39c6c44864d17dfe0; no voters:
I20260812 03:55:28.872182 2924 leader_election.cc:288] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:55:28.872215 2932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:28.872321 2932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 1 LEADER]: Becoming Leader. State: Replica: 6aedbf489d004bf39c6c44864d17dfe0, State: Running, Role: LEADER
I20260812 03:55:28.872407 2932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:55:28.872463 2924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:28.872733 2932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6aedbf489d004bf39c6c44864d17dfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } } }
I20260812 03:55:28.872897 2932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:28.873131 2932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aedbf489d004bf39c6c44864d17dfe0. Latest consensus state: current_term: 1 leader_uuid: "6aedbf489d004bf39c6c44864d17dfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } } }
I20260812 03:55:28.873239 2932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:28.873600 2939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:55:28.874258 2939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:55:28.876084 2939 catalog_manager.cc:1347] Generated new cluster ID: 74e66806de7448489b236d251717aee5
I20260812 03:55:28.876145 2939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:55:28 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:28.881960 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:28.870190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47870 (local address 127.17.54.254:33161)
0812 03:55:28.870417 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:28.870426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:28.870493 (+ 67us) server_negotiation.cc:407] Connection header received
0812 03:55:28.870644 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:28.870651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:28.870719 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:28.870847 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:28.871705 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:28.873065 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:28.874227 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:28.874875 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:28.874996 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:28.878025 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:28.878029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:28.878029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:28.878042 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:28.879998 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:28.880435 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:28.880438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:28.880439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:28.880560 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:28.880803 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:28.880803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:28.880804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:28.881100 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:28.881242 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:28.881599 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 03:55:28.881745 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":135,"threads_started":1}
I20260812 03:55:28.885665 2939 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:55:28.886422 2939 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:55:28.895051 2939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Generated new TSK 0
I20260812 03:55:28.895246 2939 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:55:28.929399 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33161
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:29.016224 2953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:29.016479 2953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:29.016521 2953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:29.017720 2953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:29.017794 2953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:29.017840 2953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:55:29.019407 2953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:29.019690 2953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:29.019949 2953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:29.025488 2959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:29.025532 2960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:29.025496 2963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:29.025514 2953 server_base.cc:1034] running on GCE node
I20260812 03:55:29.025951 2953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:29.026239 2953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:29.027413 2953 hybrid_clock.cc:648] HybridClock initialized: now 1786506929027404 us; error 34 us; skew 500 ppm
Aug 12 03:55:29 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:29.030535 2953 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:55:29.031088 2953 webserver.cc:460] Webserver started at http://127.17.54.193:38151/ using document root <none> and password file <none>
I20260812 03:55:29.031335 2953 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:29.031407 2953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:29.031582 2953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:29.032701 2953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "de6a5e04ab5b4c629604255a9e7f7b8c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "934017fd80a91e7cee08fb327c05c107"
server_key_iv: "4a4965fd83e4b3f9a761224586915373"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.033092 2953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "de6a5e04ab5b4c629604255a9e7f7b8c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "934017fd80a91e7cee08fb327c05c107"
server_key_iv: "4a4965fd83e4b3f9a761224586915373"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.034579 2953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:55:29.035517 2971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.035724 2953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:29.035830 2953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "de6a5e04ab5b4c629604255a9e7f7b8c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "934017fd80a91e7cee08fb327c05c107"
server_key_iv: "4a4965fd83e4b3f9a761224586915373"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.035915 2953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:29.049070 2953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:29.051698 2953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:29.051955 2953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:29.052191 2953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:29.052477 2953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:29.052510 2953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.052532 2953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:29.052544 2953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.058938 2953 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:46867
I20260812 03:55:29.058988 3099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:46867 every 8 connection(s)
I20260812 03:55:29.059329 2953 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:55:29.065495 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 2953
I20260812 03:55:29.065605 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:55:29.065819 17627 external_mini_cluster.cc:1428] Setting key b96a3dd7aa833456c422d118562feb2d
Aug 12 03:55:29 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:29.069703 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.060092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:40739 (local address 127.17.54.254:33161)
0812 03:55:29.060165 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.060167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.060411 (+ 244us) server_negotiation.cc:407] Connection header received
0812 03:55:29.060966 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.060967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.060986 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.061038 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:29.062141 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.062743 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.063799 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.064030 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.064113 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.066658 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:29.066660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:29.066660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:29.066671 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:29.068231 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:29.068667 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:29.068668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:29.068669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:29.068735 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:29.069000 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:29.069001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:29.069001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:29.069126 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:29.069205 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.069503 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.069607 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:55:29.069906 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.060086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:40739)
0812 03:55:29.060277 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.060321 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:29.060847 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:29.061116 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:29.061119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:29.061815 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:29.062031 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.062036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.062864 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.062867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.063694 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.063696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.064134 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.064135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.064261 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.064778 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:29.064787 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:29.066555 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:29.068344 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:29.068350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:29.068353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:29.068589 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:29.068811 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:29.068812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:29.068812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:29.068930 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:29.069196 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:29.069204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:29.069360 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 03:55:29.069481 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 03:55:29.069652 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":95,"threads_started":1}
I20260812 03:55:29.070482 3100 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33161
I20260812 03:55:29.070576 3100 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:29.070839 3100 heartbeater.cc:511] Master 127.17.54.254:33161 requested a full tablet report, sending...
I20260812 03:55:29.071427 2885 ts_manager.cc:194] Registered new tserver with Master: de6a5e04ab5b4c629604255a9e7f7b8c (127.17.54.193:46867)
I20260812 03:55:29.072042 2885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:40739
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:55:29.093901 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33161
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:29.177098 3105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:29.177341 3105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:29.177383 3105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:29.178568 3105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:29.178637 3105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:29.178681 3105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:55:29.180168 3105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:29.180449 3105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:29.180687 3105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:29.185861 3111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:29.185927 3113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:29.185941 3110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:29.186131 3105 server_base.cc:1034] running on GCE node
I20260812 03:55:29.186398 3105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:29.186658 3105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:29.187832 3105 hybrid_clock.cc:648] HybridClock initialized: now 1786506929187828 us; error 41 us; skew 500 ppm
Aug 12 03:55:29 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:29.190501 3105 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:55:29.190950 3105 webserver.cc:460] Webserver started at http://127.17.54.194:36171/ using document root <none> and password file <none>
I20260812 03:55:29.191184 3105 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:29.191252 3105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:29.191414 3105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:29.192488 3105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "20999477a0fd4f7aa0d7a0047bad022d"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "347a32d788a6e516425601e6e85d4946"
server_key_iv: "907b065575b8f872725ae262ed0dc362"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.192858 3105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "20999477a0fd4f7aa0d7a0047bad022d"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "347a32d788a6e516425601e6e85d4946"
server_key_iv: "907b065575b8f872725ae262ed0dc362"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.194279 3105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:55:29.195236 3121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.195479 3105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:29.195588 3105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "20999477a0fd4f7aa0d7a0047bad022d"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "347a32d788a6e516425601e6e85d4946"
server_key_iv: "907b065575b8f872725ae262ed0dc362"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.195691 3105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:29.208649 3105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:29.210772 3105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:29.210954 3105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:29.211222 3105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:29.211573 3105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:29.211635 3105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.211686 3105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:29.211714 3105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.218197 3105 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:46555
I20260812 03:55:29.218273 3250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:46555 every 8 connection(s)
I20260812 03:55:29.218673 3105 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:55:29.219353 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 3105
I20260812 03:55:29.219475 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:55:29.219740 17627 external_mini_cluster.cc:1428] Setting key 1e5018fda28ccf3c687c2bccc277636c
Aug 12 03:55:29 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
I20260812 03:55:29.229580 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.219749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:53533 (local address 127.17.54.254:33161)
0812 03:55:29.219828 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.219831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.220026 (+ 195us) server_negotiation.cc:407] Connection header received
0812 03:55:29.220685 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.220686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.220699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.220749 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:29.221829 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.222655 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.223690 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.224030 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.224107 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.226409 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:29.226411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:29.226411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:29.226420 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:29.228047 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:29.228436 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:29.228437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:29.228438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:29.228506 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:29.228822 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:29.228823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:29.228824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:29.228986 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:29.229064 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.229357 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.229473 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:55:29.229904 3253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.219739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:53533)
0812 03:55:29.219902 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.219948 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:29.220566 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:29.220817 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:29.220821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:29.221476 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:29.221729 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.221734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.222751 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.222752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.223600 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.223603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.224121 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.224122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.224230 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.224810 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:29.224822 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:29.226260 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:29.228130 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:29.228136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:29.228138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:29.228354 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:29.228611 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:29.228612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:29.228612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:29.228735 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:29.229078 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:29.229084 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:29.229212 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 03:55:29.229305 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 03:55:29.229619 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260812 03:55:29.230541 3251 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33161
I20260812 03:55:29.230656 3251 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:29.230863 3251 heartbeater.cc:511] Master 127.17.54.254:33161 requested a full tablet report, sending...
I20260812 03:55:29.231424 2885 ts_manager.cc:194] Registered new tserver with Master: 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555)
I20260812 03:55:29.231884 2885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:53533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:55:29.250206 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33161
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:29.334666 3256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:29.334887 3256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:29.334996 3256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:29.336243 3256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:29.336287 3256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:29.336314 3256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:55:29.337857 3256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:29.338124 3256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:29.338407 3256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:55:29.343997 3256 server_base.cc:1034] running on GCE node
W20260812 03:55:29.344038 3263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:29.343978 3265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:29.343956 3262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:29.344344 3256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:29.344672 3256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:29.345840 3256 hybrid_clock.cc:648] HybridClock initialized: now 1786506929345828 us; error 47 us; skew 500 ppm
Aug 12 03:55:29 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:29.348562 3256 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:55:29.349017 3256 webserver.cc:460] Webserver started at http://127.17.54.195:32905/ using document root <none> and password file <none>
I20260812 03:55:29.349280 3256 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:29.349352 3256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:29.349509 3256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:55:29.350669 3256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "25b6fa21ab6042e89f927e6d39d5e41c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "a121d813f25259e42b9bad73f0ca3f76"
server_key_iv: "efea0a75d898e5b93375089180fa8259"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.351097 3256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "25b6fa21ab6042e89f927e6d39d5e41c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "a121d813f25259e42b9bad73f0ca3f76"
server_key_iv: "efea0a75d898e5b93375089180fa8259"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.352559 3256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:55:29.353410 3272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.353626 3256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:29.353729 3256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "25b6fa21ab6042e89f927e6d39d5e41c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "a121d813f25259e42b9bad73f0ca3f76"
server_key_iv: "efea0a75d898e5b93375089180fa8259"
server_key_version: "encryptionkey@0"
I20260812 03:55:29.353818 3256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:29.379385 3256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:29.381837 3256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:29.382014 3256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:29.382280 3256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:29.382660 3256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:29.382717 3256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.382755 3256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:29.382784 3256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:29.388851 3256 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:41143
I20260812 03:55:29.388900 3406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:41143 every 8 connection(s)
I20260812 03:55:29.389235 3256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:55:29 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:29.396620 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 3256
I20260812 03:55:29.396730 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:55:29.396914 17627 external_mini_cluster.cc:1428] Setting key 8b0bf239d87873ce01b18759dae0155c
I20260812 03:55:29.399799 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.390017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:50585 (local address 127.17.54.254:33161)
0812 03:55:29.390102 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.390105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.390275 (+ 170us) server_negotiation.cc:407] Connection header received
0812 03:55:29.390856 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.390858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.390872 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.390944 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:29.391985 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.392568 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.393585 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.393792 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.393855 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.396324 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:29.396325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:29.396326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:29.396335 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:29.398266 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:29.398762 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:29.398764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:29.398764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:29.398830 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:29.399096 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:29.399097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:29.399097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:29.399227 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:29.399304 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.399592 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.399679 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:55:29.399999 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.389953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:50585)
0812 03:55:29.390147 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.390184 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:29.390719 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:29.391030 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:29.391034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:29.391677 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:29.391879 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.391884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.392655 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.392656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.393501 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.393503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.393871 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.393872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.393999 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.394610 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:29.394619 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:29.396214 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:29.398387 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:29.398396 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:29.398398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:29.398685 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:29.398899 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:29.398900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:29.398900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:29.399021 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:29.399294 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:29.399301 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:29.399453 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 03:55:29.399567 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:55:29.399738 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
I20260812 03:55:29.400520 3407 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33161
I20260812 03:55:29.400621 3407 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:29.400768 3407 heartbeater.cc:511] Master 127.17.54.254:33161 requested a full tablet report, sending...
I20260812 03:55:29.401145 2885 ts_manager.cc:194] Registered new tserver with Master: 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
I20260812 03:55:29.401548 2885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:50585
I20260812 03:55:29.409147 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:55:29.417168 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.410639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.17.54.254:33161)
0812 03:55:29.410713 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.410715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.410780 (+ 65us) server_negotiation.cc:407] Connection header received
0812 03:55:29.410817 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.410818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.410827 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.410879 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:29.411431 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.411943 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.412726 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.412938 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.412998 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.414482 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:29.414483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:29.414484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:29.414494 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:29.415852 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:29.416186 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:29.416187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:29.416187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:29.416243 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:29.416479 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:29.416480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:29.416480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:29.416648 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:29.416744 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.416933 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.417008 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:55:29.419054 2885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:55:29.420693 2885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:55:29.428016 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.423334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46867 (local address 127.0.0.1:43520)
0812 03:55:29.423528 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.423570 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:29.423642 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:29.424152 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:29.424156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:29.424158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:29.424228 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.424230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.425606 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.425607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.426777 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.426780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.427727 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.427728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.427800 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.427806 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:55:29.427838 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 03:55:29.427927 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":70,"thread_start_us":54,"threads_started":1}
I20260812 03:55:29.428077 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.423577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:41143 (local address 127.0.0.1:40272)
0812 03:55:29.424239 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.424249 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:29.424329 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:29.424583 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:29.424584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:29.424586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:29.424622 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.424623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.425561 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.425562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.426971 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.426975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.427816 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.427817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.427876 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.427880 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:55:29.427935 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:55:29.427979 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":55,"thread_start_us":78,"threads_started":1}
I20260812 03:55:29.428073 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.423681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.17.54.195:41143)
0812 03:55:29.423866 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.423871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.424334 (+ 463us) server_negotiation.cc:407] Connection header received
0812 03:55:29.424404 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.424408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.424424 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.424495 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:29.424710 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.425436 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.427060 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.427748 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.427816 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.427888 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.427946 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.427993 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
I20260812 03:55:29.428105 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.423536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.17.54.193:46867)
0812 03:55:29.423994 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.423999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.424009 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:55:29.424049 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.424050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.424065 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.424139 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:29.424310 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.425484 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.426883 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.427617 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.427898 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.427960 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.427982 (+ 22us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.428024 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":83,"threads_started":1}
I20260812 03:55:29.428853 3331 tablet_service.cc:1467] Processing CreateTablet for tablet 41824c12a02a4a9c82bb2422247bc39b (DEFAULT_TABLE table=test-table [id=d1fb9670eb7d4fa5ba8bcdcb1fd8407a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:29.429004 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.423361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:46555 (local address 127.0.0.1:42376)
0812 03:55:29.423753 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.423763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:29.423801 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:29.424414 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:29.424415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:29.424418 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:29.424520 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.424522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.425762 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.425781 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.426975 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.426976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.427662 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.427663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.427759 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.427774 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 03:55:29.427843 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:55:29.428940 (+ 1097us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":25,"thread_start_us":102,"threads_started":1}
I20260812 03:55:29.429003 3027 tablet_service.cc:1467] Processing CreateTablet for tablet 41824c12a02a4a9c82bb2422247bc39b (DEFAULT_TABLE table=test-table [id=d1fb9670eb7d4fa5ba8bcdcb1fd8407a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:29.429262 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.423750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.17.54.194:46555)
0812 03:55:29.423927 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.423931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.423943 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:55:29.424033 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.424035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.424052 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.424139 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:29.424588 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.425645 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.427079 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.427562 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.427663 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.427729 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.429124 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.429177 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260812 03:55:29.429262 3331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41824c12a02a4a9c82bb2422247bc39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:29.429375 3027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41824c12a02a4a9c82bb2422247bc39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:29.429952 3177 tablet_service.cc:1467] Processing CreateTablet for tablet 41824c12a02a4a9c82bb2422247bc39b (DEFAULT_TABLE table=test-table [id=d1fb9670eb7d4fa5ba8bcdcb1fd8407a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:29.430261 3177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41824c12a02a4a9c82bb2422247bc39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:29.432179 3427 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Bootstrap starting.
I20260812 03:55:29.432703 3428 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: Bootstrap starting.
I20260812 03:55:29.432883 3427 tablet_bootstrap.cc:654] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:29.433426 3428 tablet_bootstrap.cc:654] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:29.433470 3427 log.cc:824] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:29.433745 3430 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Bootstrap starting.
I20260812 03:55:29.434046 3428 log.cc:824] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:29.434458 3430 tablet_bootstrap.cc:654] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:29.434561 3427 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: No bootstrap required, opened a new log
I20260812 03:55:29.434664 3427 ts_tablet_manager.cc:1397] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:55:29.435017 3430 log.cc:824] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:29.435221 3428 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: No bootstrap required, opened a new log
I20260812 03:55:29.435343 3428 ts_tablet_manager.cc:1397] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:55:29.435782 3430 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: No bootstrap required, opened a new log
I20260812 03:55:29.435864 3430 ts_tablet_manager.cc:1397] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:55:29.436769 3427 raft_consensus.cc:348] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.436978 3427 raft_consensus.cc:374] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:29.437021 3427 raft_consensus.cc:729] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6a5e04ab5b4c629604255a9e7f7b8c, State: Initialized, Role: FOLLOWER
I20260812 03:55:29.437177 3427 consensus_queue.cc:260] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.437309 3428 raft_consensus.cc:348] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.437475 3427 ts_tablet_manager.cc:1428] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:55:29.437346 3430 raft_consensus.cc:348] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.437493 3428 raft_consensus.cc:374] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:29.437530 3428 raft_consensus.cc:729] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25b6fa21ab6042e89f927e6d39d5e41c, State: Initialized, Role: FOLLOWER
I20260812 03:55:29.437609 3100 heartbeater.cc:503] Master 127.17.54.254:33161 was elected leader, sending a full tablet report...
I20260812 03:55:29.437675 3428 consensus_queue.cc:260] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.437532 3430 raft_consensus.cc:374] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:29.437940 3430 raft_consensus.cc:729] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20999477a0fd4f7aa0d7a0047bad022d, State: Initialized, Role: FOLLOWER
I20260812 03:55:29.437970 3428 ts_tablet_manager.cc:1428] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:55:29.438020 3407 heartbeater.cc:503] Master 127.17.54.254:33161 was elected leader, sending a full tablet report...
I20260812 03:55:29.438131 3430 consensus_queue.cc:260] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.438439 3430 ts_tablet_manager.cc:1428] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:55:29.438628 3251 heartbeater.cc:503] Master 127.17.54.254:33161 was elected leader, sending a full tablet report...
W20260812 03:55:29.469440 3252 tablet.cc:2369] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:55:29.560091 3101 tablet.cc:2369] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:55:29.573926 3434 raft_consensus.cc:482] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:55:29.574057 3434 raft_consensus.cc:504] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.574550 3434 leader_election.cc:288] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
I20260812 03:55:29.578859 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.574813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:41331 (local address 127.17.54.194:46555)
0812 03:55:29.574903 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.574905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.574995 (+ 90us) server_negotiation.cc:407] Connection header received
0812 03:55:29.575150 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.575151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.575167 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.575237 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:29.575465 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.576238 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.577840 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.578472 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.578552 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.578561 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.578740 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.578783 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:55:29.578859 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.575078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:40135 (local address 127.17.54.195:41143)
0812 03:55:29.575129 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.575132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.575141 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:55:29.575172 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.575173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.575183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.575237 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:29.575467 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.576238 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.577842 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.578472 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.578552 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.578561 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.578743 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.578783 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:55:29.578934 3439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.574963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:41143 (local address 127.17.54.193:40135)
0812 03:55:29.575064 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.575075 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:29.575120 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:29.575257 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:29.575258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:29.575259 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:29.575389 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.575391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.576328 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.576329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.577734 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.577738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.578566 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.578568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.578653 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.578657 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:55:29.578721 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:55:29.578789 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":98,"thread_start_us":46,"threads_started":1}
I20260812 03:55:29.578934 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.574735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:46555 (local address 127.17.54.193:41331)
0812 03:55:29.574903 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.574917 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:29.575014 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:29.575256 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:29.575257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:29.575259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:29.575381 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.575383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.576330 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.576331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.577734 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:29.577738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.578569 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.578569 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:29.578653 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.578659 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:55:29.578721 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:55:29.578789 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":123,"thread_start_us":106,"threads_started":1}
I20260812 03:55:29.579358 3355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41824c12a02a4a9c82bb2422247bc39b" candidate_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" is_pre_election: true
I20260812 03:55:29.579406 3200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41824c12a02a4a9c82bb2422247bc39b" candidate_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20999477a0fd4f7aa0d7a0047bad022d" is_pre_election: true
I20260812 03:55:29.579559 3200 raft_consensus.cc:2454] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de6a5e04ab5b4c629604255a9e7f7b8c in term 0.
I20260812 03:55:29.579560 3355 raft_consensus.cc:2454] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de6a5e04ab5b4c629604255a9e7f7b8c in term 0.
I20260812 03:55:29.579797 2973 leader_election.cc:302] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25b6fa21ab6042e89f927e6d39d5e41c, de6a5e04ab5b4c629604255a9e7f7b8c; no voters:
I20260812 03:55:29.579962 3434 raft_consensus.cc:2781] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:55:29.580044 3434 raft_consensus.cc:482] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:55:29.580082 3434 raft_consensus.cc:3037] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:29.580844 3434 raft_consensus.cc:504] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.580993 3434 leader_election.cc:288] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 1 election: Requested vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
I20260812 03:55:29.581188 3200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41824c12a02a4a9c82bb2422247bc39b" candidate_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20999477a0fd4f7aa0d7a0047bad022d"
I20260812 03:55:29.581215 3355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41824c12a02a4a9c82bb2422247bc39b" candidate_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25b6fa21ab6042e89f927e6d39d5e41c"
I20260812 03:55:29.581283 3200 raft_consensus.cc:3037] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:29.581290 3355 raft_consensus.cc:3037] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:29.582077 3200 raft_consensus.cc:2454] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de6a5e04ab5b4c629604255a9e7f7b8c in term 1.
I20260812 03:55:29.582079 3355 raft_consensus.cc:2454] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de6a5e04ab5b4c629604255a9e7f7b8c in term 1.
I20260812 03:55:29.582248 2973 leader_election.cc:302] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20999477a0fd4f7aa0d7a0047bad022d, de6a5e04ab5b4c629604255a9e7f7b8c; no voters:
I20260812 03:55:29.582374 3434 raft_consensus.cc:2781] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:29.582562 3434 raft_consensus.cc:686] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 LEADER]: Becoming Leader. State: Replica: de6a5e04ab5b4c629604255a9e7f7b8c, State: Running, Role: LEADER
I20260812 03:55:29.582684 3434 consensus_queue.cc:237] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:29.583600 2883 catalog_manager.cc:5495] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c reported cstate change: term changed from 0 to 1, leader changed from <none> to de6a5e04ab5b4c629604255a9e7f7b8c (127.17.54.193). New cstate: current_term: 1 leader_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } health_report { overall_health: UNKNOWN } } }
I20260812 03:55:29.613230 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.610517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43522 (local address 127.17.54.193:46867)
0812 03:55:29.610603 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:29.610606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:29.610616 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:55:29.610659 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:29.610660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:29.610693 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:29.610761 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:55:29.610955 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.611527 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.612325 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:29.612527 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:29.612595 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:29.612821 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:55:29.612935 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:55:29.613005 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:29.613078 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:55:29.613100 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:55:29.615281 3355 raft_consensus.cc:1261] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 1 FOLLOWER]: Refusing update from remote peer de6a5e04ab5b4c629604255a9e7f7b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:29.615281 3200 raft_consensus.cc:1261] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 1 FOLLOWER]: Refusing update from remote peer de6a5e04ab5b4c629604255a9e7f7b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:29.615753 3440 consensus_queue.cc:1035] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:29.615908 3434 consensus_queue.cc:1035] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:29.618611 3443 mvcc.cc:205] Tried to move back new op lower bound from 7317532383701393408 to 7317532383571406848. Current Snapshot: MvccSnapshot[applied={T|T < 7317532383701393408}]
I20260812 03:55:29.619666 3445 mvcc.cc:205] Tried to move back new op lower bound from 7317532383701393408 to 7317532383571406848. Current Snapshot: MvccSnapshot[applied={T|T < 7317532383701393408 or (T in {7317532383701393408})}]
I20260812 03:55:29.619835 3444 mvcc.cc:205] Tried to move back new op lower bound from 7317532383701393408 to 7317532383571406848. Current Snapshot: MvccSnapshot[applied={T|T < 7317532383701393408}]
I20260812 03:55:29.621671 2884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:29.621757 2884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:29.622550 2884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Sending DeleteTablet for 3 replicas of tablet 41824c12a02a4a9c82bb2422247bc39b
I20260812 03:55:29.622920 3027 tablet_service.cc:1514] Processing DeleteTablet for tablet 41824c12a02a4a9c82bb2422247bc39b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:29 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43520
I20260812 03:55:29.622994 3177 tablet_service.cc:1514] Processing DeleteTablet for tablet 41824c12a02a4a9c82bb2422247bc39b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:29 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:42376
I20260812 03:55:29.623018 3331 tablet_service.cc:1514] Processing DeleteTablet for tablet 41824c12a02a4a9c82bb2422247bc39b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:29 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:40272
I20260812 03:55:29.623200 3455 tablet_replica.cc:330] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: stopping tablet replica
I20260812 03:55:29.623306 3455 raft_consensus.cc:2229] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:29.623377 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 2953
I20260812 03:55:29.623415 3455 raft_consensus.cc:2258] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:29.626077 3455 ts_tablet_manager.cc:1905] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:29.627889 3456 tablet_replica.cc:330] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: stopping tablet replica
I20260812 03:55:29.628052 3456 raft_consensus.cc:2229] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:29.628206 3456 raft_consensus.cc:2258] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:29.628647 3456 ts_tablet_manager.cc:1905] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:29.630085 3456 ts_tablet_manager.cc:1918] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:29.630189 3456 log.cc:1196] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/41824c12a02a4a9c82bb2422247bc39b
I20260812 03:55:29.630561 3456 ts_tablet_manager.cc:1939] T 41824c12a02a4a9c82bb2422247bc39b P 25b6fa21ab6042e89f927e6d39d5e41c: Deleting consensus metadata
I20260812 03:55:29.631034 2869 catalog_manager.cc:4841] TS 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143): tablet 41824c12a02a4a9c82bb2422247bc39b (table test-table [id=d1fb9670eb7d4fa5ba8bcdcb1fd8407a]) successfully deleted
W20260812 03:55:29.633028 2868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:46867 (error 108)
I20260812 03:55:29.633167 3455 ts_tablet_manager.cc:1918] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:29.633276 3455 log.cc:1196] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/41824c12a02a4a9c82bb2422247bc39b
W20260812 03:55:29.633625 3423 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:55:29.633428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46867 (local address 127.0.0.1:43528)
0812 03:55:29.633524 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.633575 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:46867: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 03:55:29.633635 3455 ts_tablet_manager.cc:1939] T 41824c12a02a4a9c82bb2422247bc39b P 20999477a0fd4f7aa0d7a0047bad022d: Deleting consensus metadata
W20260812 03:55:29.633754 2868 catalog_manager.cc:4584] TS de6a5e04ab5b4c629604255a9e7f7b8c (127.17.54.193:46867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41824c12a02a4a9c82bb2422247bc39b: Network error: Client connection negotiation failed: client connection to 127.17.54.193:46867: connect: Connection reset by peer (error 104)
I20260812 03:55:29.633802 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 3105
I20260812 03:55:29.641803 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 3256
I20260812 03:55:29.641927 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:29.641777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:46555 (local address 127.0.0.1:42384)
0812 03:55:29.641844 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:29.641877 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.194:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 03:55:29.650395 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 2846
I20260812 03:55:49.660346 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33161
--webserver_interface=127.17.54.254
--webserver_port=37679
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:33161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:55:49.750393 3458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:49.750730 3458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:49.750784 3458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:49.752105 3458 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:55:49.752170 3458 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:55:49.752197 3458 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:55:49.752233 3458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:55:49.752264 3458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:49.752300 3458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:49.752331 3458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:55:49.752363 3458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:55:49.754145 3458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:33161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=37679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:49.754513 3458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:49.754782 3458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:49.764914 3468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:49.764918 3465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:49.764914 3466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:49.765247 3458 server_base.cc:1034] running on GCE node
I20260812 03:55:49.765560 3458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:49.765924 3458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:49.767081 3458 hybrid_clock.cc:648] HybridClock initialized: now 1786506949767059 us; error 40 us; skew 500 ppm
Aug 12 03:55:49 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506949, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:49.776752 3458 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:55:49.777235 3458 webserver.cc:460] Webserver started at http://127.17.54.254:37679/ using document root <none> and password file <none>
I20260812 03:55:49.777529 3458 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:49.777614 3458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:49.779211 3458 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:49.780011 3476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:49.780213 3458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:49.780350 3458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "6aedbf489d004bf39c6c44864d17dfe0"
format_stamp: "Formatted at 2026-08-12 03:55:28 on dist-test-slave-tm1g"
server_key: "4b35b77a209f9b3f3833444b266b3d2b"
server_key_iv: "93de43ebf4edbf7c293beff6123d72b9"
server_key_version: "encryptionkey@0"
I20260812 03:55:49.780792 3458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:49.789402 3458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:49.791781 3458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:49.791940 3458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:49.796700 3458 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:33161
I20260812 03:55:49.796767 3535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:33161 every 8 connection(s)
I20260812 03:55:49.797175 3458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:55:49.799022 3536 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:55:49.799727 3536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Bootstrap starting.
I20260812 03:55:49.807044 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 3458
I20260812 03:55:49.807371 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:46867
--local_ip_for_outbound_sockets=127.17.54.193
--tserver_master_addrs=127.17.54.254:33161
--webserver_port=38151
--webserver_interface=127.17.54.193
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:49.810249 3536 log.cc:824] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:49.812573 3536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:49.812932 3536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Bootstrap complete.
I20260812 03:55:49.815874 3536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:55:49.816174 3536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aedbf489d004bf39c6c44864d17dfe0, State: Initialized, Role: FOLLOWER
I20260812 03:55:49.816401 3536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:55:49.816498 3536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:55:49.816565 3536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:55:49.816615 3536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:55:49.817796 3536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:55:49.817927 3536 leader_election.cc:302] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aedbf489d004bf39c6c44864d17dfe0; no voters:
I20260812 03:55:49.818157 3536 leader_election.cc:288] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:55:49.818200 3540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:55:49.818356 3540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 2 LEADER]: Becoming Leader. State: Replica: 6aedbf489d004bf39c6c44864d17dfe0, State: Running, Role: LEADER
I20260812 03:55:49.818504 3536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:55:49.818518 3540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:55:49.819223 3542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aedbf489d004bf39c6c44864d17dfe0. Latest consensus state: current_term: 2 leader_uuid: "6aedbf489d004bf39c6c44864d17dfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } } }
I20260812 03:55:49.819321 3542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:49.819670 3549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:55:49.819717 3541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6aedbf489d004bf39c6c44864d17dfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } } }
I20260812 03:55:49.819802 3541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:55:49.820528 3549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:55:49.820618 3549 catalog_manager.cc:1259] Loaded cluster ID: 74e66806de7448489b236d251717aee5
I20260812 03:55:49.820640 3549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:55:49.821879 3549 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:55:49.822270 3549 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Loaded TSK: 0
I20260812 03:55:49.837985 3549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Generated new TSK 1
I20260812 03:55:49.838124 3549 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:55:49.908587 3538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:49.909037 3538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:49.909132 3538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:49.911250 3538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:49.911373 3538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:49.911453 3538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:55:49.913487 3538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:46867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=38151
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:49.913813 3538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:49.914145 3538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:49.921391 3566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:49.921494 3568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:49.921391 3565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:49.922693 3538 server_base.cc:1034] running on GCE node
I20260812 03:55:49.922920 3538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:49.923228 3538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:49.924394 3538 hybrid_clock.cc:648] HybridClock initialized: now 1786506949924366 us; error 40 us; skew 500 ppm
Aug 12 03:55:49 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506949, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:49.927752 3538 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:55:49.928298 3538 webserver.cc:460] Webserver started at http://127.17.54.193:38151/ using document root <none> and password file <none>
I20260812 03:55:49.928551 3538 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:49.928637 3538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:49.930076 3538 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:55:49.930902 3575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:49.931063 3538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:49.931159 3538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "de6a5e04ab5b4c629604255a9e7f7b8c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "934017fd80a91e7cee08fb327c05c107"
server_key_iv: "4a4965fd83e4b3f9a761224586915373"
server_key_version: "encryptionkey@0"
I20260812 03:55:49.931550 3538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:49.949888 3538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:49.952550 3538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:49.952742 3538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:49.953030 3538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:49.953586 3583 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:55:49.955309 3538 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:55:49.955374 3538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 03:55:49.955402 3538 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:55:49.956151 3538 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:55:49.956200 3538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:49.956250 3583 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Bootstrap starting.
I20260812 03:55:49.965015 3538 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:46867
I20260812 03:55:49.965651 3538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:55:49.966500 3712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:46867 every 8 connection(s)
I20260812 03:55:49.968936 3583 log.cc:824] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:49.971589 3583 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:55:49.971920 3583 tablet_bootstrap.cc:492] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Bootstrap complete.
I20260812 03:55:49.973054 3583 ts_tablet_manager.cc:1397] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.009s
I20260812 03:55:49.973999 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 3538
I20260812 03:55:49.974545 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:46555
--local_ip_for_outbound_sockets=127.17.54.194
--tserver_master_addrs=127.17.54.254:33161
--webserver_port=36171
--webserver_interface=127.17.54.194
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:55:49.974663 3583 raft_consensus.cc:348] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:49.974996 3583 raft_consensus.cc:729] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6a5e04ab5b4c629604255a9e7f7b8c, State: Initialized, Role: FOLLOWER
I20260812 03:55:49.975165 3583 consensus_queue.cc:260] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } }
I20260812 03:55:49.975466 3583 ts_tablet_manager.cc:1428] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Aug 12 03:55:49 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506949, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:49.983107 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:49.966481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:33517)
0812 03:55:49.969297 (+ 2816us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:49.969350 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:49.970022 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:49.970606 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:49.970610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:49.971024 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:49.971327 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:49.971334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:49.972958 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:49.972959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:49.974439 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:49.974442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:49.975483 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:49.975484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:49.975633 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:49.976242 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:49.976264 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:49.978154 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:49.981059 (+ 2905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:49.981067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:49.981069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:49.981309 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:49.981653 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:49.981654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:49.981655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:49.981777 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:49.982411 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:49.982419 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:49.982555 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 03:55:49.982651 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:55:49.982897 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2768,"thread_start_us":89,"threads_started":1}
I20260812 03:55:49.983201 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:49.966628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:33517 (local address 127.17.54.254:33161)
0812 03:55:49.969186 (+ 2558us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:49.969198 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:49.970185 (+ 987us) server_negotiation.cc:407] Connection header received
0812 03:55:49.970355 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:49.970361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:49.970415 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:49.970537 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:49.971468 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:49.972840 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:49.974556 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:49.975236 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:49.975375 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:49.978281 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:49.978286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:49.978286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:49.978325 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:49.980950 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:49.981405 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:49.981409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:49.981410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:49.981557 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:49.981858 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:49.981859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:49.981860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:49.982246 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:49.982398 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:49.982749 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 03:55:49.982934 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2495,"thread_start_us":82,"threads_started":1}
I20260812 03:55:49.984230 3713 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33161
I20260812 03:55:49.984325 3713 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:49.984521 3713 heartbeater.cc:511] Master 127.17.54.254:33161 requested a full tablet report, sending...
I20260812 03:55:49.985406 3496 ts_manager.cc:194] Registered new tserver with Master: de6a5e04ab5b4c629604255a9e7f7b8c (127.17.54.193:46867)
I20260812 03:55:49.986393 3496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:33517
I20260812 03:55:49.987437 3713 heartbeater.cc:503] Master 127.17.54.254:33161 was elected leader, sending a full tablet report...
Aug 12 03:55:49 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506949, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.193@KRBTEST.COM
I20260812 03:55:49.995559 3723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:49.986021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46867 (local address 127.0.0.1:54196)
0812 03:55:49.986236 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:49.986275 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:49.986389 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:49.986933 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:49.986939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:49.987274 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:49.987335 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:49.987338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:49.988848 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:49.988849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:49.989434 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:49.989436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:49.990115 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:49.990116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:49.990206 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:49.990785 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:49.990790 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:49.992133 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:49.994234 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:49.994242 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:49.994244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:49.994561 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:49.994816 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:49.994817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:49.994817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:49.994885 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:49.995247 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:49.995254 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:49.995304 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:55:49.995382 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:55:49.995486 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":124,"threads_started":1}
I20260812 03:55:49.995699 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:49.986538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.17.54.193:46867)
0812 03:55:49.986741 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:49.986747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:49.986761 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:55:49.986804 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:49.986807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:49.986826 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:49.986932 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:49.987432 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:49.988752 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:49.989515 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:49.990025 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:49.990101 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:49.992203 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:49.992212 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:49.992213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:49.992221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:49.994133 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:49.994661 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:49.994665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:49.994665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:49.994737 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:49.994962 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:49.994963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:49.994964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:49.995159 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:49.995248 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:49.995451 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 03:55:49.995562 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260812 03:55:49.996249 3637 tablet_service.cc:1514] Processing DeleteTablet for tablet 41824c12a02a4a9c82bb2422247bc39b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:29 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:54196
I20260812 03:55:49.996807 3725 tablet_replica.cc:330] stopping tablet replica
I20260812 03:55:49.996904 3725 raft_consensus.cc:2229] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:49.996991 3725 raft_consensus.cc:2258] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:49.997310 3725 ts_tablet_manager.cc:1905] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:49.999650 3725 ts_tablet_manager.cc:1918] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:49.999718 3725 log.cc:1196] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal/wals/41824c12a02a4a9c82bb2422247bc39b
I20260812 03:55:49.999977 3725 ts_tablet_manager.cc:1939] T 41824c12a02a4a9c82bb2422247bc39b P de6a5e04ab5b4c629604255a9e7f7b8c: Deleting consensus metadata
I20260812 03:55:50.000591 3477 catalog_manager.cc:4841] TS de6a5e04ab5b4c629604255a9e7f7b8c (127.17.54.193:46867): tablet 41824c12a02a4a9c82bb2422247bc39b (table test-table [id=d1fb9670eb7d4fa5ba8bcdcb1fd8407a]) successfully deleted
W20260812 03:55:50.071242 3721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:50.071513 3721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:50.071556 3721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:50.072914 3721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:50.073016 3721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:50.073066 3721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:55:50.074828 3721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:46555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=36171
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:50.075131 3721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:50.075399 3721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:50.080857 3731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:50.081081 3721 server_base.cc:1034] running on GCE node
W20260812 03:55:50.081137 3732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:50.080888 3735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:50.081391 3721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:50.081775 3721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:50.082937 3721 hybrid_clock.cc:648] HybridClock initialized: now 1786506950082917 us; error 42 us; skew 500 ppm
Aug 12 03:55:50 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:50.085798 3721 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:55:50.086413 3721 webserver.cc:460] Webserver started at http://127.17.54.194:36171/ using document root <none> and password file <none>
I20260812 03:55:50.086683 3721 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:50.086776 3721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:50.088295 3721 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:55:50.089067 3744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:50.089308 3721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:50.089480 3721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "20999477a0fd4f7aa0d7a0047bad022d"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "347a32d788a6e516425601e6e85d4946"
server_key_iv: "907b065575b8f872725ae262ed0dc362"
server_key_version: "encryptionkey@0"
I20260812 03:55:50.089876 3721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:50.122298 3721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:50.124560 3721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:50.124742 3721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:50.125020 3721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:50.125367 3721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:50.125422 3721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:50.125522 3721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:50.125550 3721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:50.131861 3721 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:46555
I20260812 03:55:50.131901 3871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:46555 every 8 connection(s)
I20260812 03:55:50.132265 3721 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:55:50.132699 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 3721
I20260812 03:55:50.133064 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:41143
--local_ip_for_outbound_sockets=127.17.54.195
--tserver_master_addrs=127.17.54.254:33161
--webserver_port=32905
--webserver_interface=127.17.54.195
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:55:50 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:50.144080 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.133213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:45875 (local address 127.17.54.254:33161)
0812 03:55:50.133462 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.133465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.133558 (+ 93us) server_negotiation.cc:407] Connection header received
0812 03:55:50.134407 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.134409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.134428 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.134541 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:50.135783 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.136318 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.137914 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.138170 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.138261 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.140926 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:50.140927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:50.140927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:50.140936 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:50.142619 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:50.143077 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:50.143078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:50.143078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:50.143146 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:50.143397 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:50.143398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:50.143399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:50.143528 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:50.143598 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.143875 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.143980 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212}
I20260812 03:55:50.144304 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.133208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:45875)
0812 03:55:50.133405 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:50.133455 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:50.134237 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:50.134624 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:50.134628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:50.135403 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:50.135685 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.135690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.136421 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.136423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.137799 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.137803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.138284 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.138286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.138499 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.139061 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:50.139074 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:50.140799 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:50.142724 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:50.142732 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:50.142735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:50.142985 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:50.143216 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:50.143216 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:50.143217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:50.143328 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:50.143600 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:50.143607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:50.143749 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 03:55:50.143856 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:55:50.144035 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260812 03:55:50.144809 3872 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33161
I20260812 03:55:50.144937 3872 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:50.145172 3872 heartbeater.cc:511] Master 127.17.54.254:33161 requested a full tablet report, sending...
I20260812 03:55:50.145715 3496 ts_manager.cc:194] Registered new tserver with Master: 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555)
I20260812 03:55:50.146121 3496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:45875
W20260812 03:55:50.220793 3875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:55:50.221052 3875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:55:50.221082 3875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:55:50.222405 3875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:55:50.222455 3875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:55:50.222486 3875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:55:50.224072 3875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:41143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=32905
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:55:50.224344 3875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:55:50.224597 3875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:55:50.230000 3883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:55:50.229991 3886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:50.230127 3875 server_base.cc:1034] running on GCE node
W20260812 03:55:50.229991 3881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:55:50.230448 3875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:55:50.230723 3875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:55:50.231899 3875 hybrid_clock.cc:648] HybridClock initialized: now 1786506950231885 us; error 29 us; skew 500 ppm
Aug 12 03:55:50 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:55:50.234635 3875 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:55:50.235136 3875 webserver.cc:460] Webserver started at http://127.17.54.195:32905/ using document root <none> and password file <none>
I20260812 03:55:50.235381 3875 fs_manager.cc:359] Metadata directory not provided
I20260812 03:55:50.235438 3875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:55:50.236874 3875 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:55:50.237522 3893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:50.237809 3875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:50.237923 3875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "25b6fa21ab6042e89f927e6d39d5e41c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "a121d813f25259e42b9bad73f0ca3f76"
server_key_iv: "efea0a75d898e5b93375089180fa8259"
server_key_version: "encryptionkey@0"
I20260812 03:55:50.238375 3875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:55:50.250864 3875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:55:50.253059 3875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:55:50.253235 3875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:55:50.253486 3875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:55:50.253846 3875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:55:50.253906 3875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:50.253954 3875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:55:50.253986 3875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:55:50.260836 3875 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:41143
I20260812 03:55:50.260880 4018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:41143 every 8 connection(s)
I20260812 03:55:50.261302 3875 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:55:50 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:55:50.270565 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 3875
I20260812 03:55:50.271796 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.262140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:45185 (local address 127.17.54.254:33161)
0812 03:55:50.262193 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.262196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.262477 (+ 281us) server_negotiation.cc:407] Connection header received
0812 03:55:50.262996 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.262997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.263009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.263078 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:50.264162 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.264663 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.265784 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.266000 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.266075 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.268491 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:50.268493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:50.268494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:50.268502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:50.269979 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:50.270562 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:50.270563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:50.270563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:50.270645 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:50.270994 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:50.270996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:50.270996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:50.271171 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:50.271243 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.271650 (+ 407us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.271730 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:55:50.272080 4021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.262117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:45185)
0812 03:55:50.262337 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:50.262381 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:50.262900 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:50.263157 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:50.263161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:50.263774 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:55:50.264022 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.264029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.264772 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.264774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.265670 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.265672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.266111 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.266112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.266235 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.266912 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:55:50.266921 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:55:50.268377 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:55:50.270103 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:50.270111 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:50.270113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:50.270464 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:50.270752 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:55:50.270753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:55:50.270753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:55:50.270890 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:55:50.271247 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:55:50.271254 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:55:50.271499 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 03:55:50.271616 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:55:50.271804 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":107,"threads_started":1}
I20260812 03:55:50.272655 4019 heartbeater.cc:347] Connected to a master server at 127.17.54.254:33161
I20260812 03:55:50.272804 4019 heartbeater.cc:464] Registering TS with master...
I20260812 03:55:50.273029 4019 heartbeater.cc:511] Master 127.17.54.254:33161 requested a full tablet report, sending...
I20260812 03:55:50.273437 3496 ts_manager.cc:194] Registered new tserver with Master: 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
I20260812 03:55:50.273928 3496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:45185
I20260812 03:55:50.278038 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.271009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52580 (local address 127.17.54.254:33161)
0812 03:55:50.271173 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.271175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.271278 (+ 103us) server_negotiation.cc:407] Connection header received
0812 03:55:50.271376 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.271377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.271388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.271441 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:50.272213 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.272904 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.273948 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.274229 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.274279 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.275525 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:50.275526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:50.275527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:50.275534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:50.276898 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:50.277366 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:50.277367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:50.277367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:50.277454 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:50.277617 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:50.277617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:50.277617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:50.277719 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:50.277872 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.277900 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.277966 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260812 03:55:50.279107 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:55:50.281980 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.279535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.17.54.254:33161)
0812 03:55:50.279612 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.279613 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.279735 (+ 122us) server_negotiation.cc:407] Connection header received
0812 03:55:50.279781 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.279782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.279813 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.279876 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:55:50.280097 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.280610 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.281379 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.281588 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.281650 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.281774 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:55:50.281815 (+ 41us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 03:55:50.281934 (+ 119us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52584: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 03:55:50.282073 4023 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52584: token expired
W20260812 03:55:50.282258 3412 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.54.254:33161)
I20260812 03:55:50.288273 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.282546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52592 (local address 127.17.54.254:33161)
0812 03:55:50.282586 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.282588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.282615 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:55:50.282639 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.282640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.282647 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.282685 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:55:50.283146 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.283592 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.284335 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.284517 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.284568 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.285724 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:55:50.285726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:55:50.285727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:55:50.285733 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:55:50.287268 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:50.287594 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:50.287595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:50.287595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:50.287646 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:55:50.287859 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:55:50.287860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:55:50.287860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:55:50.287957 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:55:50.288049 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.288150 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.288211 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:55:50.289537 3496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:55:50.291306 3496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:55:50.293694 3637 tablet_service.cc:1467] Processing CreateTablet for tablet 92c3dc72a8634891bf06422b24a5308a (DEFAULT_TABLE table=test-table [id=fa8a17b77af34207918f4fbd66bfce37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:50.293924 3637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c3dc72a8634891bf06422b24a5308a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:50.295676 3583 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Bootstrap starting.
I20260812 03:55:50.296027 3583 tablet_bootstrap.cc:654] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:50.296754 3583 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: No bootstrap required, opened a new log
I20260812 03:55:50.296820 3583 ts_tablet_manager.cc:1397] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:55:50.297206 3583 raft_consensus.cc:348] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.297283 3583 raft_consensus.cc:374] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:50.297328 3583 raft_consensus.cc:729] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6a5e04ab5b4c629604255a9e7f7b8c, State: Initialized, Role: FOLLOWER
I20260812 03:55:50.297377 3583 consensus_queue.cc:260] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.297411 3723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.293294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:46555 (local address 127.0.0.1:52856)
0812 03:55:50.293370 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:50.293382 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:50.293424 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:50.293966 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:50.293967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:50.293968 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:50.294032 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.294033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.295143 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.295144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.296089 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.296092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.297130 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.297131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.297205 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.297229 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:55:50.297291 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:55:50.297339 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 03:55:50.297435 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.293520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.17.54.194:46555)
0812 03:55:50.293785 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.293790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.293802 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:55:50.293843 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.293846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.293863 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.293945 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:50.294178 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.295057 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.296403 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.297047 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.297148 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.297216 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.297306 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.297349 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":106,"threads_started":1}
I20260812 03:55:50.297461 3583 ts_tablet_manager.cc:1428] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:55:50.297911 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.293412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:41143 (local address 127.0.0.1:56260)
0812 03:55:50.293578 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:50.293587 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:50.293629 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:50.294354 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:50.294356 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:50.294358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:50.294412 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.294413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.295730 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.295732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.296729 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.296733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.297669 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.297670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.297736 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.297740 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:55:50.297796 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:55:50.297831 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":62,"threads_started":1}
I20260812 03:55:50.298215 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.293520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.17.54.195:41143)
0812 03:55:50.294154 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.294159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.294171 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:55:50.294208 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.294210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.294225 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.294495 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:50.294587 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.295621 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.296836 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.297581 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.297660 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.297717 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.298076 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.298123 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":102,"threads_started":1}
I20260812 03:55:50.298157 3801 tablet_service.cc:1467] Processing CreateTablet for tablet 92c3dc72a8634891bf06422b24a5308a (DEFAULT_TABLE table=test-table [id=fa8a17b77af34207918f4fbd66bfce37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:50.298527 3801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c3dc72a8634891bf06422b24a5308a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:50.301203 4031 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Bootstrap starting.
I20260812 03:55:50.301811 3947 tablet_service.cc:1467] Processing CreateTablet for tablet 92c3dc72a8634891bf06422b24a5308a (DEFAULT_TABLE table=test-table [id=fa8a17b77af34207918f4fbd66bfce37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:55:50.302049 4031 tablet_bootstrap.cc:654] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:50.302151 3947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c3dc72a8634891bf06422b24a5308a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:55:50.302657 4031 log.cc:824] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:50.303396 4031 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: No bootstrap required, opened a new log
I20260812 03:55:50.303452 4031 ts_tablet_manager.cc:1397] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:55:50.304903 4033 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Bootstrap starting.
I20260812 03:55:50.304819 4031 raft_consensus.cc:348] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.305042 4031 raft_consensus.cc:374] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:50.305096 4031 raft_consensus.cc:729] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20999477a0fd4f7aa0d7a0047bad022d, State: Initialized, Role: FOLLOWER
I20260812 03:55:50.305234 4031 consensus_queue.cc:260] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.305565 4031 ts_tablet_manager.cc:1428] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:55:50.305583 3872 heartbeater.cc:503] Master 127.17.54.254:33161 was elected leader, sending a full tablet report...
I20260812 03:55:50.305725 4033 tablet_bootstrap.cc:654] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Neither blocks nor log segments found. Creating new log.
I20260812 03:55:50.306416 4033 log.cc:824] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:55:50.307140 4033 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: No bootstrap required, opened a new log
I20260812 03:55:50.307225 4033 ts_tablet_manager.cc:1397] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:55:50.308571 4033 raft_consensus.cc:348] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.308737 4033 raft_consensus.cc:374] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:55:50.308787 4033 raft_consensus.cc:729] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25b6fa21ab6042e89f927e6d39d5e41c, State: Initialized, Role: FOLLOWER
I20260812 03:55:50.308918 4033 consensus_queue.cc:260] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.309209 4033 ts_tablet_manager.cc:1428] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:55:50.309262 4019 heartbeater.cc:503] Master 127.17.54.254:33161 was elected leader, sending a full tablet report...
W20260812 03:55:50.382927 3873 tablet.cc:2369] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:55:50.467203 3716 tablet.cc:2369] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:55:50.511941 4020 tablet.cc:2369] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:55:50.537985 3720 raft_consensus.cc:482] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:55:50.538146 3720 raft_consensus.cc:504] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.538614 3720 leader_election.cc:288] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
I20260812 03:55:50.543174 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.539016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:45419 (local address 127.17.54.194:46555)
0812 03:55:50.539094 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.539096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.539160 (+ 64us) server_negotiation.cc:407] Connection header received
0812 03:55:50.539292 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.539294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.539309 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.539379 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:50.539565 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.540411 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.542011 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.542682 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.542769 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.542779 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.543029 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.543095 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:55:50.543174 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.539204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:54425 (local address 127.17.54.195:41143)
0812 03:55:50.539265 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.539268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.539279 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:55:50.539374 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.539375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.539385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.539446 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:55:50.539580 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.540436 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.542012 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.542682 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.542770 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.542779 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.543046 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.543103 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:55:50.543186 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.539120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:41143 (local address 127.17.54.193:54425)
0812 03:55:50.539224 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:50.539234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:50.539281 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:50.539454 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:50.539455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:50.539456 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:50.539507 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.539508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.540527 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.540528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.541875 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.541878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.542802 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.542804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.542896 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.542932 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:55:50.543014 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:55:50.543080 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":64,"thread_start_us":44,"threads_started":1}
I20260812 03:55:50.543308 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.538893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:46555 (local address 127.17.54.193:45419)
0812 03:55:50.539069 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:50.539085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:55:50.539197 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:55:50.539401 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:55:50.539402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:55:50.539405 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:55:50.539474 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.539477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.540527 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.540528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.541875 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:55:50.541878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.542812 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.542813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:55:50.542889 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.542930 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:55:50.543009 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:55:50.543080 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":46,"thread_start_us":86,"threads_started":1}
I20260812 03:55:50.543681 3823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c3dc72a8634891bf06422b24a5308a" candidate_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20999477a0fd4f7aa0d7a0047bad022d" is_pre_election: true
I20260812 03:55:50.543691 3967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c3dc72a8634891bf06422b24a5308a" candidate_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" is_pre_election: true
I20260812 03:55:50.543853 3967 raft_consensus.cc:2454] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de6a5e04ab5b4c629604255a9e7f7b8c in term 0.
I20260812 03:55:50.543849 3823 raft_consensus.cc:2454] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de6a5e04ab5b4c629604255a9e7f7b8c in term 0.
I20260812 03:55:50.544102 3578 leader_election.cc:302] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20999477a0fd4f7aa0d7a0047bad022d, de6a5e04ab5b4c629604255a9e7f7b8c; no voters:
I20260812 03:55:50.544253 3720 raft_consensus.cc:2781] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:55:50.544339 3720 raft_consensus.cc:482] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:55:50.544377 3720 raft_consensus.cc:3037] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:50.545080 3720 raft_consensus.cc:504] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.545233 3720 leader_election.cc:288] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 1 election: Requested vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
I20260812 03:55:50.545460 3823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c3dc72a8634891bf06422b24a5308a" candidate_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20999477a0fd4f7aa0d7a0047bad022d"
I20260812 03:55:50.545569 3967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c3dc72a8634891bf06422b24a5308a" candidate_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25b6fa21ab6042e89f927e6d39d5e41c"
I20260812 03:55:50.545569 3823 raft_consensus.cc:3037] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:50.545635 3967 raft_consensus.cc:3037] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:55:50.546533 3967 raft_consensus.cc:2454] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de6a5e04ab5b4c629604255a9e7f7b8c in term 1.
I20260812 03:55:50.546533 3823 raft_consensus.cc:2454] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de6a5e04ab5b4c629604255a9e7f7b8c in term 1.
I20260812 03:55:50.546757 3578 leader_election.cc:302] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25b6fa21ab6042e89f927e6d39d5e41c, de6a5e04ab5b4c629604255a9e7f7b8c; no voters:
I20260812 03:55:50.546854 3720 raft_consensus.cc:2781] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:55:50.546995 3720 raft_consensus.cc:686] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 LEADER]: Becoming Leader. State: Replica: de6a5e04ab5b4c629604255a9e7f7b8c, State: Running, Role: LEADER
I20260812 03:55:50.547068 3720 consensus_queue.cc:237] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:55:50.547909 3496 catalog_manager.cc:5495] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c reported cstate change: term changed from 0 to 1, leader changed from <none> to de6a5e04ab5b4c629604255a9e7f7b8c (127.17.54.193). New cstate: current_term: 1 leader_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } health_report { overall_health: HEALTHY } } }
I20260812 03:55:50.581202 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.578577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.17.54.193:46867)
0812 03:55:50.578801 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 03:55:50.578802 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:55:50.578809 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:55:50.578850 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:55:50.578851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:55:50.578872 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:55:50.578961 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:55:50.579118 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.579613 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.580352 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:55:50.580595 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:55:50.580675 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:55:50.580846 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:55:50.580953 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:55:50.581035 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:55:50.581080 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 03:55:50.581102 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":132,"threads_started":1}
I20260812 03:55:50.583102 3823 raft_consensus.cc:1261] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 1 FOLLOWER]: Refusing update from remote peer de6a5e04ab5b4c629604255a9e7f7b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:50.583101 3967 raft_consensus.cc:1261] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 1 FOLLOWER]: Refusing update from remote peer de6a5e04ab5b4c629604255a9e7f7b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:55:50.583490 3720 consensus_queue.cc:1035] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:50.583657 4039 consensus_queue.cc:1035] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:55:50.586642 4043 mvcc.cc:205] Tried to move back new op lower bound from 7317532469585145856 to 7317532469441359872. Current Snapshot: MvccSnapshot[applied={T|T < 7317532469585145856}]
I20260812 03:55:50.587337 4044 mvcc.cc:205] Tried to move back new op lower bound from 7317532469585145856 to 7317532469441359872. Current Snapshot: MvccSnapshot[applied={T|T < 7317532469585145856}]
I20260812 03:55:50.587445 4042 mvcc.cc:205] Tried to move back new op lower bound from 7317532469585145856 to 7317532469441359872. Current Snapshot: MvccSnapshot[applied={T|T < 7317532469585145856 or (T in {7317532469585145856})}]
I20260812 03:55:50.588614 3496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:50.588702 3496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:55:50.589468 3496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Sending DeleteTablet for 3 replicas of tablet 92c3dc72a8634891bf06422b24a5308a
I20260812 03:55:50.589787 3637 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c3dc72a8634891bf06422b24a5308a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:50 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:54196
I20260812 03:55:50.589808 3801 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c3dc72a8634891bf06422b24a5308a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:50 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:52856
I20260812 03:55:50.589887 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 3538
I20260812 03:55:50.591564 4055 tablet_replica.cc:330] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: stopping tablet replica
I20260812 03:55:50.591691 4055 raft_consensus.cc:2229] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:50.591817 4055 raft_consensus.cc:2258] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:50.592551 4055 ts_tablet_manager.cc:1905] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:50.593129 3947 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c3dc72a8634891bf06422b24a5308a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:55:50 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:56260
I20260812 03:55:50.593514 4056 tablet_replica.cc:330] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: stopping tablet replica
I20260812 03:55:50.593652 4056 raft_consensus.cc:2229] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:55:50.593799 4056 raft_consensus.cc:2258] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:55:50.594918 4056 ts_tablet_manager.cc:1905] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:55:50.595037 4055 ts_tablet_manager.cc:1918] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:50.595116 4055 log.cc:1196] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/92c3dc72a8634891bf06422b24a5308a
I20260812 03:55:50.595371 4055 ts_tablet_manager.cc:1939] T 92c3dc72a8634891bf06422b24a5308a P 20999477a0fd4f7aa0d7a0047bad022d: Deleting consensus metadata
I20260812 03:55:50.595834 3478 catalog_manager.cc:4841] TS 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555): tablet 92c3dc72a8634891bf06422b24a5308a (table test-table [id=fa8a17b77af34207918f4fbd66bfce37]) successfully deleted
I20260812 03:55:50.597193 4056 ts_tablet_manager.cc:1918] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:55:50.597307 4056 log.cc:1196] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/92c3dc72a8634891bf06422b24a5308a
I20260812 03:55:50.597689 4056 ts_tablet_manager.cc:1939] T 92c3dc72a8634891bf06422b24a5308a P 25b6fa21ab6042e89f927e6d39d5e41c: Deleting consensus metadata
I20260812 03:55:50.598212 3478 catalog_manager.cc:4841] TS 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143): tablet 92c3dc72a8634891bf06422b24a5308a (table test-table [id=fa8a17b77af34207918f4fbd66bfce37]) successfully deleted
W20260812 03:55:50.603852 3477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:46867 (error 108)
I20260812 03:55:50.604242 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 3721
I20260812 03:55:50.607573 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:55:50.607384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:46867 (local address 127.0.0.1:54212)
0812 03:55:50.607477 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 03:55:50.607535 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:46867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 03:55:50.607735 3477 catalog_manager.cc:4584] TS de6a5e04ab5b4c629604255a9e7f7b8c (127.17.54.193:46867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92c3dc72a8634891bf06422b24a5308a: Network error: Client connection negotiation failed: client connection to 127.17.54.193:46867: connect: Connection refused (error 111)
I20260812 03:55:50.612188 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 3875
I20260812 03:55:50.623170 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 3458
I20260812 03:56:10.631915 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33161
--webserver_interface=127.17.54.254
--webserver_port=37679
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:33161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:10.720824 4059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:10.721066 4059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:10.721092 4059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:10.722381 4059 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:56:10.722434 4059 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:56:10.722447 4059 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:56:10.722460 4059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:10.722473 4059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:10.722486 4059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:10.722499 4059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:10.722559 4059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:10.724201 4059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:33161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=37679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:10.724502 4059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:10.724784 4059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:10.730908 4064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:10.730902 4059 server_base.cc:1034] running on GCE node
W20260812 03:56:10.730871 4068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:10.730847 4065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:10.731547 4059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:10.731935 4059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:10.733150 4059 hybrid_clock.cc:648] HybridClock initialized: now 1786506970733141 us; error 44 us; skew 500 ppm
Aug 12 03:56:10 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506970, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:10.736156 4059 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:10.736696 4059 webserver.cc:460] Webserver started at http://127.17.54.254:37679/ using document root <none> and password file <none>
I20260812 03:56:10.736941 4059 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:10.737015 4059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:10.738802 4059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:56:10.739610 4075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:10.739893 4059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:10.740020 4059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "6aedbf489d004bf39c6c44864d17dfe0"
format_stamp: "Formatted at 2026-08-12 03:55:28 on dist-test-slave-tm1g"
server_key: "4b35b77a209f9b3f3833444b266b3d2b"
server_key_iv: "93de43ebf4edbf7c293beff6123d72b9"
server_key_version: "encryptionkey@0"
I20260812 03:56:10.740532 4059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:10.768034 4059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:10.770682 4059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:10.770895 4059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:10.775290 4059 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:33161
I20260812 03:56:10.775324 4129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:33161 every 8 connection(s)
I20260812 03:56:10.775792 4059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:10.777406 4130 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:56:10.778119 4130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Bootstrap starting.
I20260812 03:56:10.779340 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 4059
I20260812 03:56:10.779728 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:46867
--local_ip_for_outbound_sockets=127.17.54.193
--tserver_master_addrs=127.17.54.254:33161
--webserver_port=38151
--webserver_interface=127.17.54.193
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:56:10.788038 4130 log.cc:824] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:10.790532 4130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:56:10.790885 4130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Bootstrap complete.
I20260812 03:56:10.793406 4130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:56:10.793673 4130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aedbf489d004bf39c6c44864d17dfe0, State: Initialized, Role: FOLLOWER
I20260812 03:56:10.793900 4130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:56:10.794008 4130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:10.794065 4130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:10.794113 4130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:56:10.795233 4130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:56:10.795360 4130 leader_election.cc:302] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aedbf489d004bf39c6c44864d17dfe0; no voters:
I20260812 03:56:10.795589 4130 leader_election.cc:288] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:56:10.795629 4135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:56:10.795753 4135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [term 3 LEADER]: Becoming Leader. State: Replica: 6aedbf489d004bf39c6c44864d17dfe0, State: Running, Role: LEADER
I20260812 03:56:10.795881 4130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:10.795929 4135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } }
I20260812 03:56:10.796317 4136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6aedbf489d004bf39c6c44864d17dfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } } }
I20260812 03:56:10.796334 4137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aedbf489d004bf39c6c44864d17dfe0. Latest consensus state: current_term: 3 leader_uuid: "6aedbf489d004bf39c6c44864d17dfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aedbf489d004bf39c6c44864d17dfe0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 33161 } } }
I20260812 03:56:10.796428 4136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:10.796458 4137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:10.797109 4144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:10.798043 4144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:10.798161 4144 catalog_manager.cc:1259] Loaded cluster ID: 74e66806de7448489b236d251717aee5
I20260812 03:56:10.798198 4144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:10.799614 4144 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:10.800083 4144 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Loaded TSK: 0
I20260812 03:56:10.800138 4144 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6aedbf489d004bf39c6c44864d17dfe0: Loaded TSK: 1
F20260812 03:56:10.800446 4144 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fae4e00f9ad google::LogMessage::Fail() at ??:0
@ 0x7fae4e01288a google::LogMessage::SendToLog() at ??:0
@ 0x7fae4e00f4de google::LogMessage::Flush() at ??:0
@ 0x7fae4e010a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fae5077c104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fae50779d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fae55f691cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fae55f6948e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fae55f69594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fae55f2fc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fae55f39ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fae4fedd0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fae4fed2954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fae50d366db start_thread at ??:0
@ 0x7fae4cb0d61f clone at ??:0
W20260812 03:56:10.900027 4133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:10.900252 4133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:10.900280 4133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:10.901613 4133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:10.901662 4133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:10.901693 4133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:56:10.903494 4133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:46867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=38151
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:10.903771 4133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:10.904081 4133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:10.909688 4175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:10.909672 4178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:10.909821 4133 server_base.cc:1034] running on GCE node
W20260812 03:56:10.909931 4176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:10.910197 4133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:10.910550 4133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:10.911748 4133 hybrid_clock.cc:648] HybridClock initialized: now 1786506970911732 us; error 33 us; skew 500 ppm
Aug 12 03:56:10 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506970, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:10.914686 4133 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:56:10.915234 4133 webserver.cc:460] Webserver started at http://127.17.54.193:38151/ using document root <none> and password file <none>
I20260812 03:56:10.915508 4133 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:10.915582 4133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:10.917079 4133 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:10.918109 4187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:10.918371 4133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:10.918488 4133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "de6a5e04ab5b4c629604255a9e7f7b8c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "934017fd80a91e7cee08fb327c05c107"
server_key_iv: "4a4965fd83e4b3f9a761224586915373"
server_key_version: "encryptionkey@0"
I20260812 03:56:10.918880 4133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:10.933779 4133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:10.936523 4133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:10.936738 4133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:10.937019 4133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:10.937676 4194 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:56:10.940173 4133 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:56:10.940266 4133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260812 03:56:10.940362 4133 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:56:10.941058 4133 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:56:10.941126 4133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:56:10.941207 4194 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Bootstrap starting.
I20260812 03:56:10.949115 4133 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:46867
I20260812 03:56:10.949604 4133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:56:10.949168 4314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:46867 every 8 connection(s)
I20260812 03:56:10.953395 4194 log.cc:824] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:10.956247 4194 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:56:10.956665 4194 tablet_bootstrap.cc:492] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Bootstrap complete.
I20260812 03:56:10.956694 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 4133
I20260812 03:56:10.957026 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:46555
--local_ip_for_outbound_sockets=127.17.54.194
--tserver_master_addrs=127.17.54.254:33161
--webserver_port=36171
--webserver_interface=127.17.54.194
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:56:10.957978 4194 ts_tablet_manager.cc:1397] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.011s
I20260812 03:56:10.959671 4194 raft_consensus.cc:348] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:56:10.960023 4194 raft_consensus.cc:729] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6a5e04ab5b4c629604255a9e7f7b8c, State: Initialized, Role: FOLLOWER
I20260812 03:56:10.960219 4194 consensus_queue.cc:260] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:56:10.960529 4194 ts_tablet_manager.cc:1428] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260812 03:56:11.020071 4318 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:56:10.950617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:32953)
0812 03:56:10.950815 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:10.950862 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:10.951333 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:11.019751 (+ 68418us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
W20260812 03:56:11.020323 4315 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:33161 (0 consecutive failures): Network error: Failed to ping master at 127.17.54.254:33161: Client connection negotiation failed: client connection to 127.17.54.254:33161: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:56:11.020721 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:11.020601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:49885)
0812 03:56:11.020656 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:11.020702 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:56:11.021116 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:11.021032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:45281)
0812 03:56:11.021077 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:11.021101 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:56:11.021255 4315 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:56:11.047163 4320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:11.047444 4320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:11.047488 4320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:11.048799 4320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:11.048915 4320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:11.048998 4320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:56:11.050760 4320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:46555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=36171
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:11.051064 4320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:11.051316 4320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:11.056996 4330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:11.056998 4328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:11.057065 4327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:11.057142 4320 server_base.cc:1034] running on GCE node
I20260812 03:56:11.057489 4320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:11.057807 4320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:11.058962 4320 hybrid_clock.cc:648] HybridClock initialized: now 1786506971058934 us; error 54 us; skew 500 ppm
Aug 12 03:56:11 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506971, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:11.061764 4320 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:56:11.062656 4320 webserver.cc:460] Webserver started at http://127.17.54.194:36171/ using document root <none> and password file <none>
I20260812 03:56:11.062920 4320 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:11.062971 4320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:11.064457 4320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:56:11.065233 4338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:11.065582 4320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:11.065740 4320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "20999477a0fd4f7aa0d7a0047bad022d"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "347a32d788a6e516425601e6e85d4946"
server_key_iv: "907b065575b8f872725ae262ed0dc362"
server_key_version: "encryptionkey@0"
I20260812 03:56:11.066203 4320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:11.080977 4320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:11.083374 4320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:11.083588 4320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:11.083866 4320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:11.084234 4320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:11.084301 4320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:11.084369 4320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:11.084398 4320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:11.090852 4320 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:46555
I20260812 03:56:11.090998 4461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:46555 every 8 connection(s)
I20260812 03:56:11.091327 4320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:56:11.092585 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:11.092108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:39759)
0812 03:56:11.092326 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:11.092405 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
W20260812 03:56:11.093119 4462 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:33161 (0 consecutive failures): Network error: Failed to ping master at 127.17.54.254:33161: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
I20260812 03:56:11.093484 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:11.093370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:54959)
0812 03:56:11.093434 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:11.093464 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 03:56:11.093742 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 4320
I20260812 03:56:11.094064 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:41143
--local_ip_for_outbound_sockets=127.17.54.195
--tserver_master_addrs=127.17.54.254:33161
--webserver_port=32905
--webserver_interface=127.17.54.195
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:56:11.094679 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:11.094574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:43521)
0812 03:56:11.094625 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:11.094655 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 03:56:11.094803 4462 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:56:11.181401 4465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:11.181617 4465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:11.181643 4465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:11.182843 4465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:11.182888 4465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:11.182916 4465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:56:11.184435 4465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:41143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=32905
--tserver_master_addrs=127.17.54.254:33161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:11.184710 4465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:11.184968 4465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:11.190793 4470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:11.190793 4473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:11.190793 4471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:11.190871 4465 server_base.cc:1034] running on GCE node
I20260812 03:56:11.191236 4465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:11.191507 4465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:11.192662 4465 hybrid_clock.cc:648] HybridClock initialized: now 1786506971192646 us; error 35 us; skew 500 ppm
Aug 12 03:56:11 dist-test-slave-tm1g krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506971, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:11.195465 4465 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:56:11.195955 4465 webserver.cc:460] Webserver started at http://127.17.54.195:32905/ using document root <none> and password file <none>
I20260812 03:56:11.196218 4465 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:11.196265 4465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:11.197688 4465 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:11.198447 4480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:11.198643 4465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:56:11.198720 4465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "25b6fa21ab6042e89f927e6d39d5e41c"
format_stamp: "Formatted at 2026-08-12 03:55:29 on dist-test-slave-tm1g"
server_key: "a121d813f25259e42b9bad73f0ca3f76"
server_key_iv: "efea0a75d898e5b93375089180fa8259"
server_key_version: "encryptionkey@0"
I20260812 03:56:11.199100 4465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:11.219440 4465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:11.222043 4465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:11.222205 4465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:11.222482 4465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:11.222824 4465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:11.222882 4465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:11.222927 4465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:11.222954 4465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:11.229187 4465 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:41143
I20260812 03:56:11.229246 4604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:41143 every 8 connection(s)
I20260812 03:56:11.229628 4465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:56:11.230274 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 4465
I20260812 03:56:11.230808 4608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:11.230396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:58303)
0812 03:56:11.230582 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:11.230665 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
W20260812 03:56:11.231248 4606 heartbeater.cc:644] Failed to heartbeat to 127.17.54.254:33161 (0 consecutive failures): Network error: Failed to ping master at 127.17.54.254:33161: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
I20260812 03:56:11.231534 4608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:11.231435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:52967)
0812 03:56:11.231487 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:11.231514 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:56:11.231846 4608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:11.231768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:39521)
0812 03:56:11.231810 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:11.231831 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:56:11.232010 4606 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:56:12.021868 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:12.021634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:51797)
0812 03:56:12.021757 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:12.021796 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260812 03:56:12.095464 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:12.095175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:40107)
0812 03:56:12.095358 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:12.095404 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":63,"threads_started":1}
I20260812 03:56:12.232642 4612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:12.232393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:45811)
0812 03:56:12.232541 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:12.232584 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260812 03:56:12.344668 4613 raft_consensus.cc:482] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:12.344800 4613 raft_consensus.cc:504] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:56:12.345204 4613 leader_election.cc:288] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
Aug 12 03:56:12 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506970, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.195@KRBTEST.COM
Aug 12 03:56:12 dist-test-slave-tm1g krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506970, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.194@KRBTEST.COM
I20260812 03:56:12.356891 4616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:12.345622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:56345 (local address 127.17.54.194:46555)
0812 03:56:12.345770 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:12.345782 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:12.345819 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:56:12.346359 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:12.346362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:12.346418 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:12.346530 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:12.347365 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:12.348265 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:12.349289 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:12.349714 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:12.349846 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:12.353130 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:12.353134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:12.353135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:12.353147 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:12.355102 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:12.355658 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:12.355661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:12.355662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:12.355772 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:12.356027 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:12.356028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:12.356028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:12.356313 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:12.356404 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:12.356654 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 03:56:12.356787 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260812 03:56:12.356878 4615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:12.345628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:41143 (local address 127.17.54.193:47667)
0812 03:56:12.345737 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:12.345746 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:12.345794 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:12.346742 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:12.346743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:12.347034 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:12.347229 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:12.347235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:12.348370 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:12.348371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:12.349187 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:12.349189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:12.349821 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:12.349822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:12.349944 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:12.350658 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:12.350666 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:12.352559 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:12.354962 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:12.354971 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:12.354974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:12.355290 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:12.355650 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:12.355651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:12.355651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:12.355803 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:12.356258 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:12.356266 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:12.356423 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 03:56:12.356537 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:56:12.356762 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":8,"thread_start_us":57,"threads_started":1}
I20260812 03:56:12.356935 4618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:12.345792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:47667 (local address 127.17.54.195:41143)
0812 03:56:12.345923 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:12.345958 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:12.345986 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:56:12.346512 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:12.346516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:12.346568 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:12.346724 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:12.347389 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:12.348258 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:12.349288 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:12.349713 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:12.349853 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:12.352701 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:12.352705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:12.352706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:12.352748 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:12.354866 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:12.355399 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:12.355402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:12.355403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:12.355544 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:12.355885 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:12.355886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:12.355886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:12.356181 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:12.356243 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:12.356634 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 03:56:12.356807 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
I20260812 03:56:12.357394 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:12.345474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:46555 (local address 127.17.54.193:56345)
0812 03:56:12.345539 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:12.345556 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:12.345634 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:12.346606 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:12.346609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:12.347031 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:12.347229 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:12.347235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:12.348407 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:12.348408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:12.349188 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:12.349191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:12.349817 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:12.349818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:12.349938 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:12.350686 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:12.350690 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:12.352991 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:12.355228 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:12.355229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:12.355230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:12.355493 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:12.355879 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:12.355880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:12.355880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:12.355952 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:12.356408 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:12.356409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:12.356463 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 03:56:12.356575 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:56:12.356865 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":187}
W20260812 03:56:12.357836 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
W20260812 03:56:12.358589 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
I20260812 03:56:12.358676 4189 leader_election.cc:302] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de6a5e04ab5b4c629604255a9e7f7b8c; no voters: 20999477a0fd4f7aa0d7a0047bad022d, 25b6fa21ab6042e89f927e6d39d5e41c
I20260812 03:56:12.358876 4613 raft_consensus.cc:2726] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:56:13.022732 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:13.022466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:41599)
0812 03:56:13.022602 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:13.022645 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260812 03:56:13.096416 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:13.096136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:42235)
0812 03:56:13.096289 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:13.096369 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260812 03:56:13.233382 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:13.233183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:38489)
0812 03:56:13.233302 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:13.233341 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260812 03:56:14.023610 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:14.023346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:57939)
0812 03:56:14.023520 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:14.023562 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":71,"threads_started":1}
I20260812 03:56:14.097293 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:14.097047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:56251)
0812 03:56:14.097202 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:14.097242 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260812 03:56:14.234139 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:14.233883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:44339)
0812 03:56:14.234049 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:14.234089 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260812 03:56:14.419173 4626 raft_consensus.cc:482] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:14.419308 4626 raft_consensus.cc:504] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:56:14.419507 4626 leader_election.cc:288] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
W20260812 03:56:14.420048 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
W20260812 03:56:14.420133 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
I20260812 03:56:14.420183 4189 leader_election.cc:302] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de6a5e04ab5b4c629604255a9e7f7b8c; no voters: 20999477a0fd4f7aa0d7a0047bad022d, 25b6fa21ab6042e89f927e6d39d5e41c
I20260812 03:56:14.420277 4626 raft_consensus.cc:2726] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:56:15.024474 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:15.024224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:55603)
0812 03:56:15.024380 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:15.024424 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260812 03:56:15.098114 4629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:15.097872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:44291)
0812 03:56:15.098026 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:15.098063 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260812 03:56:15.235011 4630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:15.234766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:47269)
0812 03:56:15.234921 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:15.234964 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260812 03:56:16.025434 4631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:16.025143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:40007)
0812 03:56:16.025336 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:16.025380 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260812 03:56:16.098896 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:16.098629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:49265)
0812 03:56:16.098805 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:16.098845 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
I20260812 03:56:16.235961 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:16.235656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:42553)
0812 03:56:16.235838 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:16.235888 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":88,"threads_started":1}
I20260812 03:56:17.026429 4634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:17.026106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:35249)
0812 03:56:17.026271 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:17.026378 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
I20260812 03:56:17.099864 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:17.099532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:51549)
0812 03:56:17.099757 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:17.099814 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":74,"threads_started":1}
I20260812 03:56:17.236932 4636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:17.236636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:60967)
0812 03:56:17.236830 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:17.236879 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260812 03:56:17.281661 4637 raft_consensus.cc:482] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:17.281804 4637 raft_consensus.cc:504] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:56:17.281970 4637 leader_election.cc:288] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
W20260812 03:56:17.282334 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
W20260812 03:56:17.282409 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
I20260812 03:56:17.282425 4189 leader_election.cc:302] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de6a5e04ab5b4c629604255a9e7f7b8c; no voters: 20999477a0fd4f7aa0d7a0047bad022d, 25b6fa21ab6042e89f927e6d39d5e41c
I20260812 03:56:17.282529 4637 raft_consensus.cc:2726] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:56:18.027335 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:18.027033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:45179)
0812 03:56:18.027235 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:18.027285 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":97,"threads_started":1}
I20260812 03:56:18.100754 4639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:18.100505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:43247)
0812 03:56:18.100654 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:18.100700 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260812 03:56:18.237813 4640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:18.237544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:44943)
0812 03:56:18.237720 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:18.237760 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260812 03:56:19.028229 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:19.027943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:58517)
0812 03:56:19.028127 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:19.028177 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260812 03:56:19.101696 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:19.101443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:42863)
0812 03:56:19.101591 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:19.101643 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260812 03:56:19.216866 4643 raft_consensus.cc:482] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:19.217023 4643 raft_consensus.cc:504] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:56:19.217249 4643 leader_election.cc:288] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
W20260812 03:56:19.217720 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
W20260812 03:56:19.217803 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
I20260812 03:56:19.217820 4189 leader_election.cc:302] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de6a5e04ab5b4c629604255a9e7f7b8c; no voters: 20999477a0fd4f7aa0d7a0047bad022d, 25b6fa21ab6042e89f927e6d39d5e41c
I20260812 03:56:19.217901 4643 raft_consensus.cc:2726] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:56:19.238746 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:19.238475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:40653)
0812 03:56:19.238645 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:19.238698 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260812 03:56:20.029110 4647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:20.028856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:46677)
0812 03:56:20.029014 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:20.029061 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260812 03:56:20.102612 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:20.102278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:43123)
0812 03:56:20.102437 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:20.102521 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260812 03:56:20.239503 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:20.239265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:57497)
0812 03:56:20.239409 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:20.239451 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260812 03:56:21.011034 4650 raft_consensus.cc:482] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:21.011222 4650 raft_consensus.cc:504] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:56:21.011431 4650 leader_election.cc:288] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
W20260812 03:56:21.011915 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
W20260812 03:56:21.012010 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
I20260812 03:56:21.012032 4189 leader_election.cc:302] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de6a5e04ab5b4c629604255a9e7f7b8c; no voters: 20999477a0fd4f7aa0d7a0047bad022d, 25b6fa21ab6042e89f927e6d39d5e41c
I20260812 03:56:21.012156 4650 raft_consensus.cc:2726] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:56:21.029989 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:21.029730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:59715)
0812 03:56:21.029891 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:21.029938 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260812 03:56:21.103404 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:21.103153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:47019)
0812 03:56:21.103299 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:21.103353 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260812 03:56:21.240356 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:21.240090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:43965)
0812 03:56:21.240255 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:21.240298 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260812 03:56:22.030969 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:22.030705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:60071)
0812 03:56:22.030870 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:22.030919 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260812 03:56:22.104233 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:22.104002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:36919)
0812 03:56:22.104137 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:22.104181 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260812 03:56:22.241313 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:22.240994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:54271)
0812 03:56:22.241214 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:22.241262 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":169,"threads_started":1}
I20260812 03:56:23.031960 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:23.031642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:45625)
0812 03:56:23.031850 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:23.031906 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":69,"threads_started":1}
I20260812 03:56:23.105058 4659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:23.104810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:33831)
0812 03:56:23.104961 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:23.105009 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260812 03:56:23.242230 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:23.242001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:52007)
0812 03:56:23.242137 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:23.242180 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260812 03:56:24.032856 4661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:24.032566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:55139)
0812 03:56:24.032745 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:24.032795 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":70,"threads_started":1}
I20260812 03:56:24.105983 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:24.105732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:34027)
0812 03:56:24.105888 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:24.105933 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260812 03:56:24.243108 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:24.242872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:39843)
0812 03:56:24.243011 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:24.243054 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260812 03:56:25.033799 4665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:25.033524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:34745)
0812 03:56:25.033703 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:25.033752 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
I20260812 03:56:25.106848 4666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:25.106606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:59021)
0812 03:56:25.106743 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:25.106792 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260812 03:56:25.186015 4667 raft_consensus.cc:482] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:25.186125 4667 raft_consensus.cc:504] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20999477a0fd4f7aa0d7a0047bad022d" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 46555 } } peers { permanent_uuid: "25b6fa21ab6042e89f927e6d39d5e41c" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 41143 } } peers { permanent_uuid: "de6a5e04ab5b4c629604255a9e7f7b8c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 46867 } }
I20260812 03:56:25.186355 4667 leader_election.cc:288] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555), 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143)
W20260812 03:56:25.186834 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 25b6fa21ab6042e89f927e6d39d5e41c (127.17.54.195:41143): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
W20260812 03:56:25.186926 4189 leader_election.cc:341] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20999477a0fd4f7aa0d7a0047bad022d (127.17.54.194:46555): Not found: Tablet not found: 92c3dc72a8634891bf06422b24a5308a
I20260812 03:56:25.187001 4189 leader_election.cc:302] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de6a5e04ab5b4c629604255a9e7f7b8c; no voters: 20999477a0fd4f7aa0d7a0047bad022d, 25b6fa21ab6042e89f927e6d39d5e41c
I20260812 03:56:25.187093 4667 raft_consensus.cc:2726] T 92c3dc72a8634891bf06422b24a5308a P de6a5e04ab5b4c629604255a9e7f7b8c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:56:25.244007 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:25.243725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:50499)
0812 03:56:25.243886 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:25.243933 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260812 03:56:26.034884 4669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:26.034593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.193:41239)
0812 03:56:26.034788 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:26.034836 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":69,"threads_started":1}
I20260812 03:56:26.107724 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:26.107468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.194:38539)
0812 03:56:26.107632 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:26.107674 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":67,"threads_started":1}
I20260812 03:56:26.239797 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 4133
I20260812 03:56:26.245222 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:26.244741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:33161 (local address 127.17.54.195:41357)
0812 03:56:26.245117 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:26.245156 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.254:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":65,"threads_started":1}
I20260812 03:56:26.249974 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 4320
I20260812 03:56:26.257474 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 4465
2026-08-12T03:56:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58788 ms)
[----------] 30 tests from SecurityITest (244778 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:26 dist-test-slave-tm1g krb5kdc[4678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:26 dist-test-slave-tm1g krb5kdc[4678](info): set up 2 sockets
Aug 12 03:56:26 dist-test-slave-tm1g krb5kdc[4678](info): commencing operation
krb5kdc: starting...
W20260812 03:56:27.417419 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:27 dist-test-slave-tm1g krb5kdc[4678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:56:27.554666 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:44511
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:27.641786 4692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:27.642028 4692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:27.642056 4692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:27.643338 4692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:27.643389 4692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:27.643404 4692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:27.643417 4692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:27.643429 4692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:27.645004 4692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:44511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:44511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:27.645236 4692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:27.645462 4692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:56:27.651746 4692 server_base.cc:1034] running on GCE node
W20260812 03:56:27.651691 4701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:27.651705 4699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:27.651691 4697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:27.652186 4692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:27.652573 4692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:27.653744 4692 hybrid_clock.cc:648] HybridClock initialized: now 1786506987653734 us; error 49 us; skew 500 ppm
Aug 12 03:56:27 dist-test-slave-tm1g krb5kdc[4678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506987, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:27.656517 4692 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:27.657027 4692 webserver.cc:460] Webserver started at http://127.17.54.254:39013/ using document root <none> and password file <none>
I20260812 03:56:27.657286 4692 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:27.657336 4692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:27.657676 4692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:27.659278 4692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "38c571b62ecb4593ba657c4fabf931fa"
format_stamp: "Formatted at 2026-08-12 03:56:27 on dist-test-slave-tm1g"
server_key: "fc1293c6c34823a7b21ea53418e435ad"
server_key_iv: "bef7e56151c6ac7229d60566fc4ee997"
server_key_version: "encryptionkey@0"
I20260812 03:56:27.659804 4692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "38c571b62ecb4593ba657c4fabf931fa"
format_stamp: "Formatted at 2026-08-12 03:56:27 on dist-test-slave-tm1g"
server_key: "fc1293c6c34823a7b21ea53418e435ad"
server_key_iv: "bef7e56151c6ac7229d60566fc4ee997"
server_key_version: "encryptionkey@0"
I20260812 03:56:27.661379 4692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:27.662393 4710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:27.662671 4692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:27.662811 4692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "38c571b62ecb4593ba657c4fabf931fa"
format_stamp: "Formatted at 2026-08-12 03:56:27 on dist-test-slave-tm1g"
server_key: "fc1293c6c34823a7b21ea53418e435ad"
server_key_iv: "bef7e56151c6ac7229d60566fc4ee997"
server_key_version: "encryptionkey@0"
I20260812 03:56:27.662897 4692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:27.670567 4692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:27.672981 4692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:27.673166 4692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:27.677700 4692 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:44511
I20260812 03:56:27.677735 4769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:44511 every 8 connection(s)
I20260812 03:56:27.678105 4692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:27.679154 4770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:27.680524 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 4692
I20260812 03:56:27.680660 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:27.680930 17627 external_mini_cluster.cc:1428] Setting key d638b9ece962098d98348f1e32ce1f87
I20260812 03:56:27.682031 4770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa: Bootstrap starting.
I20260812 03:56:27.682839 4770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:27.683140 4770 log.cc:824] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:27.683889 4770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa: No bootstrap required, opened a new log
I20260812 03:56:27.685773 4770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c571b62ecb4593ba657c4fabf931fa" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44511 } }
I20260812 03:56:27.685946 4770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:27.685998 4770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c571b62ecb4593ba657c4fabf931fa, State: Initialized, Role: FOLLOWER
I20260812 03:56:27.686170 4770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c571b62ecb4593ba657c4fabf931fa" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44511 } }
I20260812 03:56:27.686275 4770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:27.686337 4770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:27.686388 4770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:27.687271 4770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c571b62ecb4593ba657c4fabf931fa" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44511 } }
I20260812 03:56:27.687403 4770 leader_election.cc:302] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38c571b62ecb4593ba657c4fabf931fa; no voters:
I20260812 03:56:27.687613 4770 leader_election.cc:288] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:27.687682 4775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:27.687875 4775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [term 1 LEADER]: Becoming Leader. State: Replica: 38c571b62ecb4593ba657c4fabf931fa, State: Running, Role: LEADER
I20260812 03:56:27.687902 4770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:27.687973 4775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c571b62ecb4593ba657c4fabf931fa" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44511 } }
I20260812 03:56:27.688333 4776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38c571b62ecb4593ba657c4fabf931fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c571b62ecb4593ba657c4fabf931fa" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44511 } } }
I20260812 03:56:27.688311 4775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38c571b62ecb4593ba657c4fabf931fa. Latest consensus state: current_term: 1 leader_uuid: "38c571b62ecb4593ba657c4fabf931fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c571b62ecb4593ba657c4fabf931fa" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 44511 } } }
I20260812 03:56:27.688495 4775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:27.688469 4776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa [sys.catalog]: This master's current role is: LEADER
Aug 12 03:56:27 dist-test-slave-tm1g krb5kdc[4678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:27.688732 4784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:27.689286 4784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:27.693089 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:27.681621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58062 (local address 127.17.54.254:44511)
0812 03:56:27.681814 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:27.681824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:27.681859 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:56:27.682470 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:27.682480 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:27.682583 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:27.682736 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:27.683561 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:27.684653 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:27.685534 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:27.686047 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:27.686158 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:27.689376 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:27.689381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:27.689381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:27.689394 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:27.691385 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:27.691839 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:27.691842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:27.691843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:27.691935 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:27.692183 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:27.692183 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:27.692184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:27.692421 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:27.692569 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:27.692723 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:56:27.692836 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":68,"threads_started":1}
I20260812 03:56:27.693835 4784 catalog_manager.cc:1347] Generated new cluster ID: e769b48654244beeb7618157c04b908c
I20260812 03:56:27.693892 4784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:27.700595 4784 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:27.701257 4784 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:27.704336 4784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa: Generated new TSK 0
I20260812 03:56:27.704558 4784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:56:27.745952 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44511
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:27.852339 4798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:27.852610 4798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:27.852645 4798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:27.854839 4798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:27.854900 4798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:27.854933 4798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:56:27.856782 4798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:27.857084 4798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:27.857424 4798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:27.863801 4806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:27.863801 4803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:27.863795 4804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:27.864336 4798 server_base.cc:1034] running on GCE node
I20260812 03:56:27.864599 4798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:27.864838 4798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:27.866007 4798 hybrid_clock.cc:648] HybridClock initialized: now 1786506987865987 us; error 30 us; skew 500 ppm
Aug 12 03:56:27 dist-test-slave-tm1g krb5kdc[4678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506987, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:27.869300 4798 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:56:27.869807 4798 webserver.cc:460] Webserver started at http://127.17.54.193:37053/ using document root <none> and password file <none>
I20260812 03:56:27.870110 4798 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:27.870193 4798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:27.870396 4798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:27.871587 4798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "19fd16f3335945b3b6defffc8283de49"
format_stamp: "Formatted at 2026-08-12 03:56:27 on dist-test-slave-tm1g"
server_key: "a5651a6b46d839ae34ab99818901e33f"
server_key_iv: "ec441f272cff2a182eeee031e9839e2d"
server_key_version: "encryptionkey@0"
I20260812 03:56:27.872469 4798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "19fd16f3335945b3b6defffc8283de49"
format_stamp: "Formatted at 2026-08-12 03:56:27 on dist-test-slave-tm1g"
server_key: "a5651a6b46d839ae34ab99818901e33f"
server_key_iv: "ec441f272cff2a182eeee031e9839e2d"
server_key_version: "encryptionkey@0"
I20260812 03:56:27.874799 4798 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 03:56:27.875859 4816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:27.876307 4798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:27.876655 4798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "19fd16f3335945b3b6defffc8283de49"
format_stamp: "Formatted at 2026-08-12 03:56:27 on dist-test-slave-tm1g"
server_key: "a5651a6b46d839ae34ab99818901e33f"
server_key_iv: "ec441f272cff2a182eeee031e9839e2d"
server_key_version: "encryptionkey@0"
I20260812 03:56:27.876780 4798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:27.897606 4798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:27.900472 4798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:27.900859 4798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:27.901203 4798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:27.901561 4798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:27.901620 4798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:27.901659 4798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:27.901705 4798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:27.908998 4798 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:44247
I20260812 03:56:27.909065 4943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:44247 every 8 connection(s)
I20260812 03:56:27.909427 4798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:56:27.915337 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 4798
I20260812 03:56:27.915442 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:56:27.915702 17627 external_mini_cluster.cc:1428] Setting key 8f4f30416cf213841e81b3aba32bc915
Aug 12 03:56:27 dist-test-slave-tm1g krb5kdc[4678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506987, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:27.923657 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:27.910414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:46869 (local address 127.17.54.254:44511)
0812 03:56:27.910541 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:27.910543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:27.910769 (+ 226us) server_negotiation.cc:407] Connection header received
0812 03:56:27.911314 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:27.911316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:27.911328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:27.911387 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:27.912371 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:27.912945 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:27.914613 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:27.914849 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:27.914925 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:27.918953 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:27.918956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:27.918956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:27.918974 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:27.921722 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:27.922287 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:27.922289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:27.922289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:27.922408 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:27.922679 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:27.922680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:27.922681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:27.922875 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:27.922959 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:27.923415 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 03:56:27.923531 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 03:56:27.923691 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:27.910421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:44511 (local address 127.17.54.193:46869)
0812 03:56:27.910652 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:27.910691 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:27.911192 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:27.911441 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:27.911444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:27.912051 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:27.912258 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:27.912263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:27.913171 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:27.913173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:27.914496 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:27.914502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:27.915044 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:27.915046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:27.915240 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:27.916480 (+ 1240us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:27.916494 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:27.918728 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:27.921883 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:27.921897 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:27.921899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:27.922177 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:27.922467 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:27.922468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:27.922468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:27.922587 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:27.922993 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:27.923001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:27.923205 (+ 204us) client_negotiation.cc:769] Sending connection context
0812 03:56:27.923332 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 03:56:27.923497 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":154,"threads_started":1}
I20260812 03:56:27.924546 4944 heartbeater.cc:347] Connected to a master server at 127.17.54.254:44511
I20260812 03:56:27.924726 4944 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:27.924974 4944 heartbeater.cc:511] Master 127.17.54.254:44511 requested a full tablet report, sending...
I20260812 03:56:27.925787 4730 ts_manager.cc:194] Registered new tserver with Master: 19fd16f3335945b3b6defffc8283de49 (127.17.54.193:44247)
I20260812 03:56:27.926882 4730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:46869
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:56:27.955681 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44511
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:28.065382 4949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:28.065881 4949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:28.065913 4949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:28.067597 4949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:28.067672 4949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:28.067706 4949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:56:28.069416 4949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:28.069734 4949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:28.070012 4949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:28.076915 4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.076915 4958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.077214 4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:28.077442 4949 server_base.cc:1034] running on GCE node
I20260812 03:56:28.077829 4949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:28.078094 4949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:28.079457 4949 hybrid_clock.cc:648] HybridClock initialized: now 1786506988079435 us; error 32 us; skew 500 ppm
Aug 12 03:56:28 dist-test-slave-tm1g krb5kdc[4678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506988, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:28.083762 4949 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:56:28.085067 4949 webserver.cc:460] Webserver started at http://127.17.54.194:37321/ using document root <none> and password file <none>
I20260812 03:56:28.085462 4949 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:28.085532 4949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:28.085680 4949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:28.088076 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "d7e4f95f26a64830af407ee5857f06d8"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "ee50584a575b97f30e879d717c319ba6"
server_key_iv: "61f4807c283536ab6a9a1da74e3b84dd"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.088585 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "d7e4f95f26a64830af407ee5857f06d8"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "ee50584a575b97f30e879d717c319ba6"
server_key_iv: "61f4807c283536ab6a9a1da74e3b84dd"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.090818 4949 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:56:28.091859 4967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.092499 4949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:28.092942 4949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "d7e4f95f26a64830af407ee5857f06d8"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "ee50584a575b97f30e879d717c319ba6"
server_key_iv: "61f4807c283536ab6a9a1da74e3b84dd"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.093189 4949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:28.129529 4949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:28.133086 4949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:28.133304 4949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:28.133594 4949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:28.134630 4949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:28.134718 4949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.134757 4949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:28.134788 4949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.147028 4949 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:36287
I20260812 03:56:28.147200 5092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:36287 every 8 connection(s)
I20260812 03:56:28.147526 4949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:56:28.154793 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 4949
I20260812 03:56:28.155223 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:56:28.155594 17627 external_mini_cluster.cc:1428] Setting key c47a72607d71bdd924adb75b561bb18c
Aug 12 03:56:28 dist-test-slave-tm1g krb5kdc[4678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506988, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:28.164502 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.148741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:46433 (local address 127.17.54.254:44511)
0812 03:56:28.148790 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.148792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.149061 (+ 269us) server_negotiation.cc:407] Connection header received
0812 03:56:28.149400 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.149401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.149412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.149482 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:28.150721 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.151245 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.152411 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.152612 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.152886 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.158468 (+ 5582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:28.158470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:28.158471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:28.158481 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:28.161497 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:28.162946 (+ 1449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:28.162947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:28.162948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:28.163020 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:28.163526 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:28.163527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:28.163527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:28.163664 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:28.163722 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.164321 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.164418 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:56:28.164812 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.148548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:44511 (local address 127.17.54.194:46433)
0812 03:56:28.148737 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.148785 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:28.149285 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:28.149655 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:28.149659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:28.150293 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:28.150532 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.150537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.151387 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.151388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.152245 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.152248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.153013 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.153015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.153264 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.154785 (+ 1521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:28.154799 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:28.158200 (+ 3401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:28.161689 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:28.161700 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:28.161703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:28.162661 (+ 958us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:28.163137 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:28.163139 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:28.163139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:28.163321 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:28.163732 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:28.163739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:28.164066 (+ 327us) client_negotiation.cc:769] Sending connection context
0812 03:56:28.164235 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 03:56:28.164479 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260812 03:56:28.165730 5093 heartbeater.cc:347] Connected to a master server at 127.17.54.254:44511
I20260812 03:56:28.165863 5093 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:28.166141 5093 heartbeater.cc:511] Master 127.17.54.254:44511 requested a full tablet report, sending...
I20260812 03:56:28.166818 4730 ts_manager.cc:194] Registered new tserver with Master: d7e4f95f26a64830af407ee5857f06d8 (127.17.54.194:36287)
I20260812 03:56:28.167331 4730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:46433
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:56:28.202412 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44511
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:28.296780 5100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:28.297070 5100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:28.297113 5100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:28.298468 5100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:28.298530 5100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:28.298578 5100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:56:28.300351 5100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:28.300647 5100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:28.300925 5100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:28.306627 5108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.306619 5106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.307011 5105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:28.307020 5100 server_base.cc:1034] running on GCE node
I20260812 03:56:28.307504 5100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:28.307827 5100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:28.309039 5100 hybrid_clock.cc:648] HybridClock initialized: now 1786506988309034 us; error 43 us; skew 500 ppm
Aug 12 03:56:28 dist-test-slave-tm1g krb5kdc[4678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506988, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:28.312021 5100 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:56:28.312549 5100 webserver.cc:460] Webserver started at http://127.17.54.195:41099/ using document root <none> and password file <none>
I20260812 03:56:28.312827 5100 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:28.312901 5100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:28.313066 5100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:28.314201 5100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "b4966adccddf4193a90be671ac42ab55"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "a866606f005f9c9900cb186563fd8c3d"
server_key_iv: "6848253f66f4d7ae1101bfa62e1b2063"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.314631 5100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "b4966adccddf4193a90be671ac42ab55"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "a866606f005f9c9900cb186563fd8c3d"
server_key_iv: "6848253f66f4d7ae1101bfa62e1b2063"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.316433 5100 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:56:28.317559 5118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.317795 5100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:28.317940 5100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "b4966adccddf4193a90be671ac42ab55"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "a866606f005f9c9900cb186563fd8c3d"
server_key_iv: "6848253f66f4d7ae1101bfa62e1b2063"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.318030 5100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:28.339974 5100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:28.342547 5100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:28.342756 5100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:28.343050 5100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:28.343437 5100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:28.343502 5100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.343554 5100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:28.343590 5100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.350553 5100 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:34253
I20260812 03:56:28.350699 5241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:34253 every 8 connection(s)
I20260812 03:56:28.351013 5100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:56:28 dist-test-slave-tm1g krb5kdc[4678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506988, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:28.359609 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 5100
I20260812 03:56:28.359745 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:56:28.359962 17627 external_mini_cluster.cc:1428] Setting key 824c4a452a75b6b32ae1324f49d7a617
I20260812 03:56:28.362275 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.351955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:34487 (local address 127.17.54.254:44511)
0812 03:56:28.352027 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.352030 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.352392 (+ 362us) server_negotiation.cc:407] Connection header received
0812 03:56:28.352926 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.352928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.352939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.353000 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:28.354083 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.354717 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.355822 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.356022 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.356107 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.358637 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:28.358638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:28.358639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:28.358647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:28.360707 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:28.361226 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:28.361228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:28.361228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:28.361300 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:28.361580 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:28.361581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:28.361581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:28.361716 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:28.361795 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.362087 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.362175 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:56:28.362512 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.351927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:44511 (local address 127.17.54.195:34487)
0812 03:56:28.352177 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.352232 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:28.352818 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:28.353066 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:28.353070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:28.353725 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:28.353979 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.353985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.354824 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.354825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.355711 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.355714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.356117 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.356118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.356227 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.356842 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:28.356851 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:28.358392 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:28.360804 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:28.360812 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:28.360815 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:28.361137 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:28.361371 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:28.361372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:28.361372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:28.361500 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:28.361786 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:28.361794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:28.361941 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 03:56:28.362062 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 03:56:28.362246 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":123,"threads_started":1}
I20260812 03:56:28.362986 5242 heartbeater.cc:347] Connected to a master server at 127.17.54.254:44511
I20260812 03:56:28.363092 5242 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:28.363250 5242 heartbeater.cc:511] Master 127.17.54.254:44511 requested a full tablet report, sending...
I20260812 03:56:28.363660 4730 ts_manager.cc:194] Registered new tserver with Master: b4966adccddf4193a90be671ac42ab55 (127.17.54.195:34253)
I20260812 03:56:28.364058 4730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:34487
I20260812 03:56:28.372479 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:56:28.381940 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.374265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.17.54.254:44511)
0812 03:56:28.374431 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.374433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.374444 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:56:28.374563 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.374564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.374575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.374635 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:28.375305 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.376291 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.377410 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.377635 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.377713 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.379102 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:28.379104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:28.379104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:28.379111 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:28.380655 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:28.381016 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:28.381017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:28.381017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:28.381086 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:28.381291 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:28.381292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:28.381292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:28.381471 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:28.381601 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.381735 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.381826 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 03:56:28.384132 4730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:56:28.386366 4730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:56:28.393450 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.389130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:44247 (local address 127.0.0.1:56012)
0812 03:56:28.389328 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.389372 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:28.389448 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:28.389779 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:28.389783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:28.389785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:28.389866 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.389868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.391029 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.391030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.392118 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.392120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.393077 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.393079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.393171 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.393187 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 03:56:28.393279 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:56:28.393353 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":56,"threads_started":1}
I20260812 03:56:28.393455 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.389245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.17.54.193:44247)
0812 03:56:28.389592 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.389597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.389608 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:56:28.389655 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.389657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.389701 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.389774 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:28.389979 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.390869 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.392216 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.392983 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.393057 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.393119 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.393304 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.393350 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":70,"threads_started":1}
I20260812 03:56:28.394057 5260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.389466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.17.54.195:34253)
0812 03:56:28.389649 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.389653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.389941 (+ 288us) server_negotiation.cc:407] Connection header received
0812 03:56:28.389992 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.389994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.390010 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.390086 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:28.390338 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.391540 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.392749 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.393514 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.393593 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.393656 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.393896 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.393953 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260812 03:56:28.393999 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.389150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:34253 (local address 127.0.0.1:49642)
0812 03:56:28.389847 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.389858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:28.389912 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:28.390161 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:28.390162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:28.390164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:28.390210 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.390212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.391660 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.391661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.392629 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.392631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.393697 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.393698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.393770 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.393776 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:56:28.393877 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:56:28.393926 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":119,"threads_started":1}
I20260812 03:56:28.394260 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.389149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:36287 (local address 127.0.0.1:59960)
0812 03:56:28.389396 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.389405 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:28.389450 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:28.390006 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:28.390007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:28.390009 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:28.390067 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.390069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.391646 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.391647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.392711 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.392713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.394030 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.394032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.394103 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.394108 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:56:28.394143 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 03:56:28.394189 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
I20260812 03:56:28.394425 4873 tablet_service.cc:1467] Processing CreateTablet for tablet 2025f1e4e63a47cf99a096beca19d4e2 (DEFAULT_TABLE table=test-table [id=846d69be2c354c37ac4483e4881d591b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:28.394738 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.389425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59960 (local address 127.17.54.194:36287)
0812 03:56:28.389620 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.389625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.389638 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:56:28.389681 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.389683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.389700 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.390167 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:28.390228 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.391513 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.392808 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.393921 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.394511 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.394589 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.394615 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.394658 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":82,"threads_started":1}
I20260812 03:56:28.395087 4873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2025f1e4e63a47cf99a096beca19d4e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:28.395654 5020 tablet_service.cc:1467] Processing CreateTablet for tablet 2025f1e4e63a47cf99a096beca19d4e2 (DEFAULT_TABLE table=test-table [id=846d69be2c354c37ac4483e4881d591b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:28.396173 5020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2025f1e4e63a47cf99a096beca19d4e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:28.398289 5261 tablet_bootstrap.cc:492] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49: Bootstrap starting.
I20260812 03:56:28.399108 5261 tablet_bootstrap.cc:654] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:28.399150 5172 tablet_service.cc:1467] Processing CreateTablet for tablet 2025f1e4e63a47cf99a096beca19d4e2 (DEFAULT_TABLE table=test-table [id=846d69be2c354c37ac4483e4881d591b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:28.399516 5263 tablet_bootstrap.cc:492] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: Bootstrap starting.
I20260812 03:56:28.399502 5172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2025f1e4e63a47cf99a096beca19d4e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:28.399807 5261 log.cc:824] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:28.400228 5263 tablet_bootstrap.cc:654] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:28.400787 5261 tablet_bootstrap.cc:492] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49: No bootstrap required, opened a new log
I20260812 03:56:28.400810 5263 log.cc:824] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:28.400866 5261 ts_tablet_manager.cc:1397] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:56:28.401729 5263 tablet_bootstrap.cc:492] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: No bootstrap required, opened a new log
I20260812 03:56:28.401798 5263 ts_tablet_manager.cc:1397] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:56:28.402899 5266 tablet_bootstrap.cc:492] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: Bootstrap starting.
I20260812 03:56:28.402956 5261 raft_consensus.cc:348] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.403132 5261 raft_consensus.cc:374] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:28.403175 5261 raft_consensus.cc:729] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fd16f3335945b3b6defffc8283de49, State: Initialized, Role: FOLLOWER
I20260812 03:56:28.403337 5261 consensus_queue.cc:260] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.403657 4944 heartbeater.cc:503] Master 127.17.54.254:44511 was elected leader, sending a full tablet report...
I20260812 03:56:28.403682 5266 tablet_bootstrap.cc:654] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:28.403649 5261 ts_tablet_manager.cc:1428] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260812 03:56:28.403887 5263 raft_consensus.cc:348] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.404073 5263 raft_consensus.cc:374] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:28.404101 5263 raft_consensus.cc:729] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4f95f26a64830af407ee5857f06d8, State: Initialized, Role: FOLLOWER
I20260812 03:56:28.404214 5263 consensus_queue.cc:260] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.404312 5266 log.cc:824] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:28.404488 5263 ts_tablet_manager.cc:1428] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:56:28.404508 5093 heartbeater.cc:503] Master 127.17.54.254:44511 was elected leader, sending a full tablet report...
I20260812 03:56:28.405524 5266 tablet_bootstrap.cc:492] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: No bootstrap required, opened a new log
I20260812 03:56:28.405611 5266 ts_tablet_manager.cc:1397] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:56:28.407086 5266 raft_consensus.cc:348] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.407265 5266 raft_consensus.cc:374] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:28.407316 5266 raft_consensus.cc:729] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4966adccddf4193a90be671ac42ab55, State: Initialized, Role: FOLLOWER
I20260812 03:56:28.407442 5266 consensus_queue.cc:260] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.407788 5266 ts_tablet_manager.cc:1428] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:56:28.407828 5242 heartbeater.cc:503] Master 127.17.54.254:44511 was elected leader, sending a full tablet report...
W20260812 03:56:28.410420 4945 tablet.cc:2369] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:56:28.444255 5272 raft_consensus.cc:482] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:28.444435 5272 raft_consensus.cc:504] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.444768 5272 leader_election.cc:288] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19fd16f3335945b3b6defffc8283de49 (127.17.54.193:44247), d7e4f95f26a64830af407ee5857f06d8 (127.17.54.194:36287)
I20260812 03:56:28.448933 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.444996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:44247 (local address 127.17.54.195:48705)
0812 03:56:28.445064 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.445078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:28.445153 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:28.445368 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:28.445369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:28.445371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:28.445457 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.445459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.446479 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.446481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.447992 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.447996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.448640 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.448641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.448740 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.448744 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:56:28.448789 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 03:56:28.448828 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":47}
I20260812 03:56:28.448974 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.445147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:48705 (local address 127.17.54.193:44247)
0812 03:56:28.445204 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.445206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.445218 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:56:28.445251 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.445252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.445268 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.445333 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:28.445556 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.446400 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.448130 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.448547 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.448613 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.448622 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.448824 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.448888 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:56:28.449011 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.445160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:39489 (local address 127.17.54.194:36287)
0812 03:56:28.445213 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.445215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.445415 (+ 200us) server_negotiation.cc:407] Connection header received
0812 03:56:28.445439 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.445440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.445449 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.445487 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:28.445654 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.446376 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.448106 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.448570 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.448673 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.448682 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.448890 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.448935 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:56:28.449074 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.444996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:36287 (local address 127.17.54.195:39489)
0812 03:56:28.445268 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.445283 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:28.445372 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:28.445510 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:28.445511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:28.445513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:28.445578 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.445579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.446479 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.446481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.447993 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.447996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.448696 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.448698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.448800 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.448806 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:56:28.448871 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:56:28.448914 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":193,"thread_start_us":128,"threads_started":1}
I20260812 03:56:28.449520 4894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2025f1e4e63a47cf99a096beca19d4e2" candidate_uuid: "b4966adccddf4193a90be671ac42ab55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fd16f3335945b3b6defffc8283de49" is_pre_election: true
I20260812 03:56:28.449532 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2025f1e4e63a47cf99a096beca19d4e2" candidate_uuid: "b4966adccddf4193a90be671ac42ab55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4f95f26a64830af407ee5857f06d8" is_pre_election: true
I20260812 03:56:28.449697 5040 raft_consensus.cc:2454] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4966adccddf4193a90be671ac42ab55 in term 0.
I20260812 03:56:28.449708 4894 raft_consensus.cc:2454] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4966adccddf4193a90be671ac42ab55 in term 0.
I20260812 03:56:28.449934 5122 leader_election.cc:302] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4966adccddf4193a90be671ac42ab55, d7e4f95f26a64830af407ee5857f06d8; no voters:
I20260812 03:56:28.450099 5272 raft_consensus.cc:2781] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:56:28.450196 5272 raft_consensus.cc:482] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:56:28.450234 5272 raft_consensus.cc:3037] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:28.451105 5272 raft_consensus.cc:504] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.451285 5272 leader_election.cc:288] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [CANDIDATE]: Term 1 election: Requested vote from peers 19fd16f3335945b3b6defffc8283de49 (127.17.54.193:44247), d7e4f95f26a64830af407ee5857f06d8 (127.17.54.194:36287)
I20260812 03:56:28.451458 4894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2025f1e4e63a47cf99a096beca19d4e2" candidate_uuid: "b4966adccddf4193a90be671ac42ab55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fd16f3335945b3b6defffc8283de49"
I20260812 03:56:28.451495 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2025f1e4e63a47cf99a096beca19d4e2" candidate_uuid: "b4966adccddf4193a90be671ac42ab55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4f95f26a64830af407ee5857f06d8"
I20260812 03:56:28.451534 4894 raft_consensus.cc:3037] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:28.451614 5040 raft_consensus.cc:3037] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:28.452503 4894 raft_consensus.cc:2454] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4966adccddf4193a90be671ac42ab55 in term 1.
I20260812 03:56:28.452618 5040 raft_consensus.cc:2454] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4966adccddf4193a90be671ac42ab55 in term 1.
I20260812 03:56:28.452726 5123 leader_election.cc:302] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fd16f3335945b3b6defffc8283de49, b4966adccddf4193a90be671ac42ab55; no voters:
I20260812 03:56:28.452845 5272 raft_consensus.cc:2781] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:28.453027 5272 raft_consensus.cc:686] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 1 LEADER]: Becoming Leader. State: Replica: b4966adccddf4193a90be671ac42ab55, State: Running, Role: LEADER
I20260812 03:56:28.453140 5272 consensus_queue.cc:237] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } }
I20260812 03:56:28.454167 4730 catalog_manager.cc:5495] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4966adccddf4193a90be671ac42ab55 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "b4966adccddf4193a90be671ac42ab55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4966adccddf4193a90be671ac42ab55" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34253 } health_report { overall_health: HEALTHY } } }
I20260812 03:56:28.472426 5260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.469820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.17.54.195:34253)
0812 03:56:28.469889 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.469891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.469902 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:56:28.470013 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.470014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.470033 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.470116 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:56:28.470278 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.470812 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.471613 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.471813 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.471893 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.472066 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:56:28.472174 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:56:28.472273 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.472292 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.472314 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:56:28.474629 4894 raft_consensus.cc:1261] T 2025f1e4e63a47cf99a096beca19d4e2 P 19fd16f3335945b3b6defffc8283de49 [term 1 FOLLOWER]: Refusing update from remote peer b4966adccddf4193a90be671ac42ab55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:28.474623 5040 raft_consensus.cc:1261] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 1 FOLLOWER]: Refusing update from remote peer b4966adccddf4193a90be671ac42ab55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:28.475018 5272 consensus_queue.cc:1035] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19fd16f3335945b3b6defffc8283de49" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 44247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:28.475193 5274 consensus_queue.cc:1035] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4f95f26a64830af407ee5857f06d8" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 36287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:28.477766 5278 mvcc.cc:205] Tried to move back new op lower bound from 7317532624788979712 to 7317532624705081344. Current Snapshot: MvccSnapshot[applied={T|T < 7317532624788979712}]
I20260812 03:56:28.478241 5280 mvcc.cc:205] Tried to move back new op lower bound from 7317532624788979712 to 7317532624705081344. Current Snapshot: MvccSnapshot[applied={T|T < 7317532624788979712}]
I20260812 03:56:28.478508 5279 mvcc.cc:205] Tried to move back new op lower bound from 7317532624788979712 to 7317532624705081344. Current Snapshot: MvccSnapshot[applied={T|T < 7317532624788979712}]
I20260812 03:56:28.480610 4730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:28.480755 4730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:28.481583 4730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38c571b62ecb4593ba657c4fabf931fa: Sending DeleteTablet for 3 replicas of tablet 2025f1e4e63a47cf99a096beca19d4e2
I20260812 03:56:28.481974 5172 tablet_service.cc:1514] Processing DeleteTablet for tablet 2025f1e4e63a47cf99a096beca19d4e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:49642
I20260812 03:56:28.481963 5020 tablet_service.cc:1514] Processing DeleteTablet for tablet 2025f1e4e63a47cf99a096beca19d4e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:59960
I20260812 03:56:28.482273 4873 tablet_service.cc:1514] Processing DeleteTablet for tablet 2025f1e4e63a47cf99a096beca19d4e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:28 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:56012
I20260812 03:56:28.482452 5289 tablet_replica.cc:330] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: stopping tablet replica
I20260812 03:56:28.482550 5289 raft_consensus.cc:2229] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:56:28.482667 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 4798
I20260812 03:56:28.482748 5289 raft_consensus.cc:2258] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:28.487192 5288 tablet_replica.cc:330] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: stopping tablet replica
I20260812 03:56:28.487378 5288 raft_consensus.cc:2229] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:28.487535 5288 raft_consensus.cc:2258] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:28.488142 5288 ts_tablet_manager.cc:1905] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:28.490236 5288 ts_tablet_manager.cc:1918] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:28.490394 5288 log.cc:1196] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/2025f1e4e63a47cf99a096beca19d4e2
I20260812 03:56:28.490694 5288 ts_tablet_manager.cc:1939] T 2025f1e4e63a47cf99a096beca19d4e2 P d7e4f95f26a64830af407ee5857f06d8: Deleting consensus metadata
I20260812 03:56:28.491210 4712 catalog_manager.cc:4841] TS d7e4f95f26a64830af407ee5857f06d8 (127.17.54.194:36287): tablet 2025f1e4e63a47cf99a096beca19d4e2 (table test-table [id=846d69be2c354c37ac4483e4881d591b]) successfully deleted
W20260812 03:56:28.493590 4714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:44247 (error 108)
I20260812 03:56:28.494144 5289 ts_tablet_manager.cc:1905] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:28.494201 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.494053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:44247 (local address 127.0.0.1:56022)
0812 03:56:28.494125 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.494170 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 03:56:28.494450 4714 catalog_manager.cc:4584] TS 19fd16f3335945b3b6defffc8283de49 (127.17.54.193:44247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2025f1e4e63a47cf99a096beca19d4e2: Network error: Client connection negotiation failed: client connection to 127.17.54.193:44247: connect: Connection refused (error 111)
I20260812 03:56:28.496057 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 4949
I20260812 03:56:28.500171 5289 ts_tablet_manager.cc:1918] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:28.500276 5289 log.cc:1196] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/2025f1e4e63a47cf99a096beca19d4e2
I20260812 03:56:28.500679 5289 ts_tablet_manager.cc:1939] T 2025f1e4e63a47cf99a096beca19d4e2 P b4966adccddf4193a90be671ac42ab55: Deleting consensus metadata
I20260812 03:56:28.501201 4711 catalog_manager.cc:4841] TS b4966adccddf4193a90be671ac42ab55 (127.17.54.195:34253): tablet 2025f1e4e63a47cf99a096beca19d4e2 (table test-table [id=846d69be2c354c37ac4483e4881d591b]) successfully deleted
I20260812 03:56:28.504683 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 5100
I20260812 03:56:28.514753 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 4692
2026-08-12T03:56:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2256 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T03:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:28Z Disabled control of system clock
I20260812 03:56:28.548372 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:38883
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:38883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:56:28.634958 5295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:28.635201 5295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:28.635226 5295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:28.636502 5295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:28.636548 5295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:28.636561 5295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:28.636574 5295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:28.636586 5295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:28.638044 5295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:38883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:38883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:28.638375 5295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:28.638717 5295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:28.644253 5301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.644236 5302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.644214 5304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:28.644687 5295 server_base.cc:1034] running on GCE node
I20260812 03:56:28.644959 5295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:28.645277 5295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:28.646441 5295 hybrid_clock.cc:648] HybridClock initialized: now 1786506988646433 us; error 37 us; skew 500 ppm
I20260812 03:56:28.648064 5295 webserver.cc:460] Webserver started at http://127.17.54.254:38827/ using document root <none> and password file <none>
I20260812 03:56:28.648370 5295 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:28.648454 5295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:28.648613 5295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:28.649771 5295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "79b7c0e379eb482d85deffd6a1fd8173"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "a851153d00095ec519f271b24abb9614"
server_key_iv: "15027cf6b826add8a6358e287eff7312"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.650137 5295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "79b7c0e379eb482d85deffd6a1fd8173"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "a851153d00095ec519f271b24abb9614"
server_key_iv: "15027cf6b826add8a6358e287eff7312"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.651678 5295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:28.652654 5311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.652894 5295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:28.653009 5295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "79b7c0e379eb482d85deffd6a1fd8173"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "a851153d00095ec519f271b24abb9614"
server_key_iv: "15027cf6b826add8a6358e287eff7312"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.653103 5295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:28.676964 5295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:28.677356 5295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:28.677547 5295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:28.681905 5295 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:38883
I20260812 03:56:28.681947 5368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:38883 every 8 connection(s)
I20260812 03:56:28.682348 5295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:28.683430 5369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:28.684430 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 5295
I20260812 03:56:28.684554 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:28.684762 17627 external_mini_cluster.cc:1428] Setting key 827b3f172a2374ef33d85b986091bc3e
I20260812 03:56:28.688025 5369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173: Bootstrap starting.
I20260812 03:56:28.688817 5369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:28.689173 5369 log.cc:824] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:28.690057 5369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173: No bootstrap required, opened a new log
I20260812 03:56:28.692075 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.685532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.17.54.254:38883)
0812 03:56:28.685776 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.685784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.685831 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:56:28.686488 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.686497 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.686769 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.686934 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:28.687515 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.688569 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.689678 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.690232 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.690416 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.691245 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:28.691249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:28.691250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:28.691469 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:28.691503 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:28.691515 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:28.691574 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.691718 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.691798 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260812 03:56:28.692742 5369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7c0e379eb482d85deffd6a1fd8173" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38883 } }
I20260812 03:56:28.692900 5369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:28.692961 5369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b7c0e379eb482d85deffd6a1fd8173, State: Initialized, Role: FOLLOWER
I20260812 03:56:28.693092 5369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7c0e379eb482d85deffd6a1fd8173" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38883 } }
I20260812 03:56:28.693173 5369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:28.693197 5369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:28.693250 5369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:28.693940 5369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7c0e379eb482d85deffd6a1fd8173" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38883 } }
I20260812 03:56:28.694058 5369 leader_election.cc:302] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79b7c0e379eb482d85deffd6a1fd8173; no voters:
I20260812 03:56:28.694278 5369 leader_election.cc:288] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:28.694402 5374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:28.694597 5374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [term 1 LEADER]: Becoming Leader. State: Replica: 79b7c0e379eb482d85deffd6a1fd8173, State: Running, Role: LEADER
I20260812 03:56:28.694612 5369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:28.694726 5374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7c0e379eb482d85deffd6a1fd8173" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38883 } }
I20260812 03:56:28.695112 5376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79b7c0e379eb482d85deffd6a1fd8173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7c0e379eb482d85deffd6a1fd8173" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38883 } } }
I20260812 03:56:28.695218 5376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:28.695132 5377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79b7c0e379eb482d85deffd6a1fd8173. Latest consensus state: current_term: 1 leader_uuid: "79b7c0e379eb482d85deffd6a1fd8173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b7c0e379eb482d85deffd6a1fd8173" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38883 } } }
I20260812 03:56:28.695377 5377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:28.695778 5381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:28.696319 5381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:28.698114 5381 catalog_manager.cc:1347] Generated new cluster ID: b39a755dd65244dbb9c2a476d5f05acf
I20260812 03:56:28.698169 5381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:28.711632 5381 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:28.712256 5381 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:28.714278 5381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173: Generated new TSK 0
I20260812 03:56:28.714530 5381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:28.718603 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:38883
--builtin_ntp_servers=127.17.54.212:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:56:28.821044 5394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:28.821389 5394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:28.821451 5394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:28.823087 5394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:28.823169 5394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:28.823223 5394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:56:28.825016 5394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:38883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:28.825387 5394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:28.825762 5394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:28.832608 5400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.832724 5401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.832942 5403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:28.833513 5394 server_base.cc:1034] running on GCE node
I20260812 03:56:28.834008 5394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:28.834364 5394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:28.835588 5394 hybrid_clock.cc:648] HybridClock initialized: now 1786506988835565 us; error 41 us; skew 500 ppm
I20260812 03:56:28.837168 5394 webserver.cc:460] Webserver started at http://127.17.54.193:33111/ using document root <none> and password file <none>
I20260812 03:56:28.837518 5394 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:28.837597 5394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:28.837764 5394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:28.839030 5394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "43c36e0e70ba4f8f95e5833d746d08a5"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "65052a19fa6cb37172ae1a78233dc77d"
server_key_iv: "eab94f04da64af80062fb37c5919fdf1"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.839426 5394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "43c36e0e70ba4f8f95e5833d746d08a5"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "65052a19fa6cb37172ae1a78233dc77d"
server_key_iv: "eab94f04da64af80062fb37c5919fdf1"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.841542 5394 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:56:28.842976 5409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.843182 5394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:56:28.843319 5394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "43c36e0e70ba4f8f95e5833d746d08a5"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "65052a19fa6cb37172ae1a78233dc77d"
server_key_iv: "eab94f04da64af80062fb37c5919fdf1"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.843415 5394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:28.863763 5394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:28.864161 5394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:28.864326 5394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:28.864603 5394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:28.864991 5394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:28.865028 5394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.865052 5394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:28.865092 5394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.871529 5394 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:33851
I20260812 03:56:28.871650 5537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:33851 every 8 connection(s)
I20260812 03:56:28.872005 5394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:56:28.876260 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 5394
I20260812 03:56:28.876375 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:56:28.876617 17627 external_mini_cluster.cc:1428] Setting key 4f2f0033d046995b588430520917ed57
I20260812 03:56:28.877898 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:38883
--builtin_ntp_servers=127.17.54.212:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:56:28.879048 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.872937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:56741 (local address 127.17.54.254:38883)
0812 03:56:28.872986 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:28.872988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:28.873263 (+ 275us) server_negotiation.cc:407] Connection header received
0812 03:56:28.873898 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:28.873900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:28.873928 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:28.874043 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:28.874917 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.875488 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.877140 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.877430 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.877536 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.878677 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:28.878679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:28.878680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:28.878686 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:28.878706 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:28.878711 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:28.878784 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:28.878880 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:56:28.878927 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:56:28.879210 5540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:28.872914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:38883 (local address 127.17.54.193:56741)
0812 03:56:28.873106 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:28.873149 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:28.873700 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:28.874140 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:28.874145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:28.874153 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:28.874745 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.874751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.875614 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.875615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.877079 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:28.877082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:28.877558 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:28.877560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:28.877737 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:28.878495 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:28.878496 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:28.878539 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:28.878577 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:28.878792 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:28.878798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:28.878801 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:28.878861 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:56:28.878957 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260812 03:56:28.879954 5538 heartbeater.cc:347] Connected to a master server at 127.17.54.254:38883
I20260812 03:56:28.880088 5538 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:28.880313 5538 heartbeater.cc:511] Master 127.17.54.254:38883 requested a full tablet report, sending...
I20260812 03:56:28.880986 5329 ts_manager.cc:194] Registered new tserver with Master: 43c36e0e70ba4f8f95e5833d746d08a5 (127.17.54.193:33851)
I20260812 03:56:28.881669 5329 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.193:56741
W20260812 03:56:28.968737 5541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:28.968969 5541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:28.968993 5541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:28.970145 5541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:28.970180 5541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:28.970206 5541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:56:28.971568 5541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:38883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:28.971796 5541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:28.972061 5541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:28.977913 5549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.978029 5546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:28.978052 5547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:28.978384 5541 server_base.cc:1034] running on GCE node
I20260812 03:56:28.978583 5541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:28.978873 5541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:28.980062 5541 hybrid_clock.cc:648] HybridClock initialized: now 1786506988980048 us; error 32 us; skew 500 ppm
I20260812 03:56:28.981474 5541 webserver.cc:460] Webserver started at http://127.17.54.194:42095/ using document root <none> and password file <none>
I20260812 03:56:28.981787 5541 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:28.981863 5541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:28.982017 5541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:28.983215 5541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "22876415e0664a098fa9ecd1ad3c5238"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "5cb996945731f53db5bfe25e36c70d34"
server_key_iv: "6da2a70758801f81085129ac32d8f95c"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.983619 5541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "22876415e0664a098fa9ecd1ad3c5238"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "5cb996945731f53db5bfe25e36c70d34"
server_key_iv: "6da2a70758801f81085129ac32d8f95c"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.985083 5541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:56:28.986020 5556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:28.986258 5541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:28.986460 5541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "22876415e0664a098fa9ecd1ad3c5238"
format_stamp: "Formatted at 2026-08-12 03:56:28 on dist-test-slave-tm1g"
server_key: "5cb996945731f53db5bfe25e36c70d34"
server_key_iv: "6da2a70758801f81085129ac32d8f95c"
server_key_version: "encryptionkey@0"
I20260812 03:56:28.986568 5541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:29.010497 5541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:29.010895 5541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:29.011075 5541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:29.011341 5541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:29.011734 5541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:29.011790 5541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:29.011833 5541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:29.011876 5541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:29.019117 5541 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:43127
I20260812 03:56:29.019164 5684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:43127 every 8 connection(s)
I20260812 03:56:29.019570 5541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:56:29.025795 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.020465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:46225 (local address 127.17.54.254:38883)
0812 03:56:29.020577 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.020580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.020806 (+ 226us) server_negotiation.cc:407] Connection header received
0812 03:56:29.021326 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.021327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.021339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.021409 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:29.022251 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.022814 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.024082 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.024290 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.024365 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.025446 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:29.025447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:29.025448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:29.025453 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:29.025477 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:29.025481 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:29.025591 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.025690 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.025721 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 03:56:29.026068 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.020397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:38883 (local address 127.17.54.194:46225)
0812 03:56:29.020619 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:29.020667 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:29.021223 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:29.021519 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:29.021523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:29.021532 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:29.022136 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.022140 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.022978 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.022979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.023974 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.023977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.024397 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.024398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.024570 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.025272 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:29.025272 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:29.025319 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:29.025345 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:29.025605 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:29.025611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:29.025613 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:56:29.025677 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:56:29.025775 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260812 03:56:29.026482 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 5541
I20260812 03:56:29.026650 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:56:29.026749 5685 heartbeater.cc:347] Connected to a master server at 127.17.54.254:38883
I20260812 03:56:29.026867 5685 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:29.026907 17627 external_mini_cluster.cc:1428] Setting key 7693bcbe7d1bdf179f95c8741ced271e
I20260812 03:56:29.027105 5685 heartbeater.cc:511] Master 127.17.54.254:38883 requested a full tablet report, sending...
I20260812 03:56:29.027621 5328 ts_manager.cc:194] Registered new tserver with Master: 22876415e0664a098fa9ecd1ad3c5238 (127.17.54.194:43127)
I20260812 03:56:29.028059 5328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.194:46225
I20260812 03:56:29.028080 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:38883
--builtin_ntp_servers=127.17.54.212:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:56:29.115859 5688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:29.116093 5688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:29.116117 5688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:29.117465 5688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:29.117516 5688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:29.117544 5688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:56:29.119055 5688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:38883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:29.119391 5688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:29.119750 5688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:29.125751 5693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:29.125758 5694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:29.125758 5696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:29.126132 5688 server_base.cc:1034] running on GCE node
I20260812 03:56:29.126543 5688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:29.126847 5688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:29.128049 5688 hybrid_clock.cc:648] HybridClock initialized: now 1786506989128009 us; error 55 us; skew 500 ppm
I20260812 03:56:29.129621 5688 webserver.cc:460] Webserver started at http://127.17.54.195:43175/ using document root <none> and password file <none>
I20260812 03:56:29.129921 5688 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:29.129999 5688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:29.130275 5688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:29.131697 5688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "03aee4c35c2c4428a6818b612e0fd5ee"
format_stamp: "Formatted at 2026-08-12 03:56:29 on dist-test-slave-tm1g"
server_key: "efb161ed68419c360b6f1db9c731f914"
server_key_iv: "39f6349a8cd360a4d32c1c58a3f6da65"
server_key_version: "encryptionkey@0"
I20260812 03:56:29.132107 5688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "03aee4c35c2c4428a6818b612e0fd5ee"
format_stamp: "Formatted at 2026-08-12 03:56:29 on dist-test-slave-tm1g"
server_key: "efb161ed68419c360b6f1db9c731f914"
server_key_iv: "39f6349a8cd360a4d32c1c58a3f6da65"
server_key_version: "encryptionkey@0"
I20260812 03:56:29.133615 5688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:29.134788 5702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:29.135032 5688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:56:29.135330 5688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "03aee4c35c2c4428a6818b612e0fd5ee"
format_stamp: "Formatted at 2026-08-12 03:56:29 on dist-test-slave-tm1g"
server_key: "efb161ed68419c360b6f1db9c731f914"
server_key_iv: "39f6349a8cd360a4d32c1c58a3f6da65"
server_key_version: "encryptionkey@0"
I20260812 03:56:29.135452 5688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:29.144459 5688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:29.144824 5688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:29.144995 5688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:29.145267 5688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:29.145646 5688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:29.145705 5688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:29.145751 5688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:29.145776 5688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:29.153304 5688 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:37133
I20260812 03:56:29.153352 5831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:37133 every 8 connection(s)
I20260812 03:56:29.153705 5688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:56:29.154630 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 5688
I20260812 03:56:29.154747 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:56:29.154968 17627 external_mini_cluster.cc:1428] Setting key c59b4bc7426bb61c21453793ed1bd33e
I20260812 03:56:29.159889 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.154659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:45977 (local address 127.17.54.254:38883)
0812 03:56:29.154718 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.154720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.155026 (+ 306us) server_negotiation.cc:407] Connection header received
0812 03:56:29.155750 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.155752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.155765 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.155811 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:29.156695 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.157272 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.158357 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.158586 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.158670 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.159614 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:29.159616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:29.159616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:29.159622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:29.159639 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:29.159642 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:29.159694 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.159778 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.159806 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:56:29.160144 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.154623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:38883 (local address 127.17.54.195:45977)
0812 03:56:29.154877 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:29.154932 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:29.155643 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:29.155951 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:29.155955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:29.155963 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:29.156569 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.156574 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.157373 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.157374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.158233 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.158236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.158681 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.158682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.158834 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.159449 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:29.159450 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:29.159481 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:29.159505 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:29.159707 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:29.159712 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:29.159714 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:56:29.159783 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:56:29.159889 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":90,"threads_started":1}
I20260812 03:56:29.160648 5832 heartbeater.cc:347] Connected to a master server at 127.17.54.254:38883
I20260812 03:56:29.160759 5832 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:29.160995 5832 heartbeater.cc:511] Master 127.17.54.254:38883 requested a full tablet report, sending...
I20260812 03:56:29.161423 5328 ts_manager.cc:194] Registered new tserver with Master: 03aee4c35c2c4428a6818b612e0fd5ee (127.17.54.195:37133)
I20260812 03:56:29.161870 5328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.195:45977
I20260812 03:56:29.166941 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:56:29.171607 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.168551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45136 (local address 127.17.54.254:38883)
0812 03:56:29.168601 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.168604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.168691 (+ 87us) server_negotiation.cc:407] Connection header received
0812 03:56:29.168746 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.168747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.168756 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.168811 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:29.168961 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.169483 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.170265 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.170584 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.170657 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.171388 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:29.171389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:29.171390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:29.171394 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:29.171408 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:29.171412 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:29.171464 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.171523 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.171544 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:56:29.173614 5328 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:56:29.175467 5328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:56:29.182268 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.178356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.17.54.193:33851)
0812 03:56:29.178533 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.178538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.178565 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:56:29.178830 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.178836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.178858 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.178975 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:29.179077 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.180161 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.181112 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.181875 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.181949 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.182048 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.182159 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.182195 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":111,"threads_started":1}
I20260812 03:56:29.182284 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.178125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33851 (local address 127.0.0.1:42882)
0812 03:56:29.178390 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:29.178401 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:29.178442 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:29.178977 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:29.178978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:29.178980 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:29.179016 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.179017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.180258 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.180259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.181005 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.181007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.181959 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.181961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.182040 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.182055 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 03:56:29.182137 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:56:29.182193 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":74,"threads_started":1}
I20260812 03:56:29.182732 5841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.177907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:37133 (local address 127.0.0.1:42718)
0812 03:56:29.178348 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:29.178358 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:29.178420 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:29.178726 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:29.178730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:29.178733 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:29.178809 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.178812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.180192 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.180193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.181459 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.181461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.182377 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.182378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.182570 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.182574 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:56:29.182617 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 03:56:29.182661 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":56,"thread_start_us":62,"threads_started":1}
I20260812 03:56:29.182749 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.178023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59862 (local address 127.17.54.194:43127)
0812 03:56:29.178264 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.178269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.178285 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:56:29.178610 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.178613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.178630 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.178685 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:29.178947 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.179722 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.181149 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.181870 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.181949 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.182048 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.182618 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.182661 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":124,"threads_started":1}
I20260812 03:56:29.183069 5842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.177907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43127 (local address 127.0.0.1:59862)
0812 03:56:29.178173 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:29.178218 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:29.178499 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:29.178785 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:29.178786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:29.178788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:29.178846 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.178848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.179804 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.179804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.181079 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.181082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.182441 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.182442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.182521 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.182525 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:56:29.182956 (+ 431us) client_negotiation.cc:240] Negotiation successful
0812 03:56:29.183007 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":68,"threads_started":1}
I20260812 03:56:29.183132 5467 tablet_service.cc:1467] Processing CreateTablet for tablet eb0b61c341374ced8690903b04a31804 (DEFAULT_TABLE table=test-table [id=df4d4d0312624eeaa7c21071045e3dee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:29.183384 5845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.178128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.17.54.195:37133)
0812 03:56:29.178562 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.178566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.178575 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:56:29.178611 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.178613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.178629 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.178702 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:29.178943 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.180101 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.181553 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.182275 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.182717 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.183086 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.183246 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.183295 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":86,"threads_started":1}
I20260812 03:56:29.183554 5467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0b61c341374ced8690903b04a31804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:29.186292 5847 tablet_bootstrap.cc:492] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5: Bootstrap starting.
I20260812 03:56:29.186843 5759 tablet_service.cc:1467] Processing CreateTablet for tablet eb0b61c341374ced8690903b04a31804 (DEFAULT_TABLE table=test-table [id=df4d4d0312624eeaa7c21071045e3dee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:29.187098 5847 tablet_bootstrap.cc:654] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:29.187083 5612 tablet_service.cc:1467] Processing CreateTablet for tablet eb0b61c341374ced8690903b04a31804 (DEFAULT_TABLE table=test-table [id=df4d4d0312624eeaa7c21071045e3dee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:29.187276 5759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0b61c341374ced8690903b04a31804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:29.187450 5612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0b61c341374ced8690903b04a31804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:29.187451 5847 log.cc:824] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:29.188392 5847 tablet_bootstrap.cc:492] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5: No bootstrap required, opened a new log
I20260812 03:56:29.188465 5847 ts_tablet_manager.cc:1397] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:56:29.190912 5849 tablet_bootstrap.cc:492] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: Bootstrap starting.
I20260812 03:56:29.191397 5847 raft_consensus.cc:348] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.191536 5847 raft_consensus.cc:374] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:29.191560 5847 raft_consensus.cc:729] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c36e0e70ba4f8f95e5833d746d08a5, State: Initialized, Role: FOLLOWER
I20260812 03:56:29.191602 5849 tablet_bootstrap.cc:654] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:29.191705 5847 consensus_queue.cc:260] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.191974 5847 ts_tablet_manager.cc:1428] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260812 03:56:29.191967 5849 log.cc:824] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:29.192131 5538 heartbeater.cc:503] Master 127.17.54.254:38883 was elected leader, sending a full tablet report...
I20260812 03:56:29.192240 5850 tablet_bootstrap.cc:492] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Bootstrap starting.
I20260812 03:56:29.192715 5849 tablet_bootstrap.cc:492] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: No bootstrap required, opened a new log
I20260812 03:56:29.192788 5849 ts_tablet_manager.cc:1397] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:56:29.193516 5850 tablet_bootstrap.cc:654] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:29.194614 5850 log.cc:824] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:29.199216 5850 tablet_bootstrap.cc:492] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: No bootstrap required, opened a new log
I20260812 03:56:29.199097 5849 raft_consensus.cc:348] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.199316 5850 ts_tablet_manager.cc:1397] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260812 03:56:29.199316 5849 raft_consensus.cc:374] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:29.199358 5849 raft_consensus.cc:729] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03aee4c35c2c4428a6818b612e0fd5ee, State: Initialized, Role: FOLLOWER
I20260812 03:56:29.199507 5849 consensus_queue.cc:260] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.199824 5849 ts_tablet_manager.cc:1428] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 03:56:29.201078 5832 heartbeater.cc:503] Master 127.17.54.254:38883 was elected leader, sending a full tablet report...
I20260812 03:56:29.202131 5850 raft_consensus.cc:348] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.202347 5850 raft_consensus.cc:374] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:29.202407 5850 raft_consensus.cc:729] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22876415e0664a098fa9ecd1ad3c5238, State: Initialized, Role: FOLLOWER
I20260812 03:56:29.202503 5850 consensus_queue.cc:260] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.202725 5850 ts_tablet_manager.cc:1428] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 03:56:29.202795 5685 heartbeater.cc:503] Master 127.17.54.254:38883 was elected leader, sending a full tablet report...
I20260812 03:56:29.264405 5851 raft_consensus.cc:482] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:29.264604 5851 raft_consensus.cc:504] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.265381 5851 leader_election.cc:288] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22876415e0664a098fa9ecd1ad3c5238 (127.17.54.194:43127), 03aee4c35c2c4428a6818b612e0fd5ee (127.17.54.195:37133)
I20260812 03:56:29.269652 5845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.265684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:34295 (local address 127.17.54.195:37133)
0812 03:56:29.265795 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.265797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.265806 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:56:29.265842 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.265843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.265875 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.265950 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:29.266221 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.267046 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.268595 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.269174 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.269262 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.269271 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.269529 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.269577 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 03:56:29.269652 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.265708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:57595 (local address 127.17.54.194:43127)
0812 03:56:29.265764 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.265766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.265883 (+ 117us) server_negotiation.cc:407] Connection header received
0812 03:56:29.265912 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.265913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.265928 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.265984 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:29.266219 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.267034 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.268594 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.269190 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.269262 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.269271 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.269540 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.269585 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:56:29.269667 5540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.265591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:37133 (local address 127.17.54.193:34295)
0812 03:56:29.265657 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:29.265669 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:29.265715 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:29.265993 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:29.265994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:29.265996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:29.266103 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.266105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.267153 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.267155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.268470 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.268474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.269303 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.269305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.269408 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.269416 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:56:29.269494 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:56:29.269555 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":15}
I20260812 03:56:29.269673 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.265609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:43127 (local address 127.17.54.193:57595)
0812 03:56:29.265793 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:29.265803 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:29.265866 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:29.265996 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:29.265997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:29.265998 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:29.266105 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.266107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.267170 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.267170 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.268471 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:29.268474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.269304 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.269305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:29.269408 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.269416 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:56:29.269527 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:56:29.269559 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":128,"thread_start_us":118,"threads_started":1}
W20260812 03:56:29.270162 5686 tablet.cc:2369] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:56:29.270107 5781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0b61c341374ced8690903b04a31804" candidate_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" is_pre_election: true
I20260812 03:56:29.270233 5632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0b61c341374ced8690903b04a31804" candidate_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22876415e0664a098fa9ecd1ad3c5238" is_pre_election: true
I20260812 03:56:29.270352 5781 raft_consensus.cc:2454] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43c36e0e70ba4f8f95e5833d746d08a5 in term 0.
I20260812 03:56:29.270391 5632 raft_consensus.cc:2454] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43c36e0e70ba4f8f95e5833d746d08a5 in term 0.
I20260812 03:56:29.270634 5412 leader_election.cc:302] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22876415e0664a098fa9ecd1ad3c5238, 43c36e0e70ba4f8f95e5833d746d08a5; no voters:
I20260812 03:56:29.270810 5851 raft_consensus.cc:2781] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:56:29.270876 5851 raft_consensus.cc:482] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:56:29.270910 5851 raft_consensus.cc:3037] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:29.271734 5851 raft_consensus.cc:504] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.271903 5851 leader_election.cc:288] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [CANDIDATE]: Term 1 election: Requested vote from peers 22876415e0664a098fa9ecd1ad3c5238 (127.17.54.194:43127), 03aee4c35c2c4428a6818b612e0fd5ee (127.17.54.195:37133)
I20260812 03:56:29.272109 5781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0b61c341374ced8690903b04a31804" candidate_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03aee4c35c2c4428a6818b612e0fd5ee"
I20260812 03:56:29.272099 5632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0b61c341374ced8690903b04a31804" candidate_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22876415e0664a098fa9ecd1ad3c5238"
I20260812 03:56:29.272217 5781 raft_consensus.cc:3037] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:29.272215 5632 raft_consensus.cc:3037] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:29.272994 5781 raft_consensus.cc:2454] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43c36e0e70ba4f8f95e5833d746d08a5 in term 1.
I20260812 03:56:29.272996 5632 raft_consensus.cc:2454] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43c36e0e70ba4f8f95e5833d746d08a5 in term 1.
I20260812 03:56:29.273208 5412 leader_election.cc:302] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22876415e0664a098fa9ecd1ad3c5238, 43c36e0e70ba4f8f95e5833d746d08a5; no voters:
I20260812 03:56:29.273360 5851 raft_consensus.cc:2781] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:29.273553 5851 raft_consensus.cc:686] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [term 1 LEADER]: Becoming Leader. State: Replica: 43c36e0e70ba4f8f95e5833d746d08a5, State: Running, Role: LEADER
I20260812 03:56:29.273661 5851 consensus_queue.cc:237] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } }
I20260812 03:56:29.274598 5328 catalog_manager.cc:5495] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43c36e0e70ba4f8f95e5833d746d08a5 (127.17.54.193). New cstate: current_term: 1 leader_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43c36e0e70ba4f8f95e5833d746d08a5" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33851 } health_report { overall_health: HEALTHY } } }
I20260812 03:56:29.302138 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.299288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.17.54.193:33851)
0812 03:56:29.299359 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:29.299361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:29.299370 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:56:29.299417 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:29.299418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:29.299440 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:29.299507 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:56:29.299803 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.300313 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.301201 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:29.301402 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:29.301465 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:29.301698 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:56:29.301849 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:56:29.301930 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:29.302023 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 03:56:29.302047 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:56:29.304339 5632 raft_consensus.cc:1261] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 1 FOLLOWER]: Refusing update from remote peer 43c36e0e70ba4f8f95e5833d746d08a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:29.304339 5781 raft_consensus.cc:1261] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 1 FOLLOWER]: Refusing update from remote peer 43c36e0e70ba4f8f95e5833d746d08a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:29.304780 5857 consensus_queue.cc:1035] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aee4c35c2c4428a6818b612e0fd5ee" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 37133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:29.305025 5851 consensus_queue.cc:1035] T eb0b61c341374ced8690903b04a31804 P 43c36e0e70ba4f8f95e5833d746d08a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22876415e0664a098fa9ecd1ad3c5238" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 43127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:29.308046 5861 mvcc.cc:205] Tried to move back new op lower bound from 7317532628187217920 to 7317532628065628160. Current Snapshot: MvccSnapshot[applied={T|T < 7317532628187217920}]
I20260812 03:56:29.309918 5328 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:29.310034 5328 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:29.310946 5328 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79b7c0e379eb482d85deffd6a1fd8173: Sending DeleteTablet for 3 replicas of tablet eb0b61c341374ced8690903b04a31804
I20260812 03:56:29.311291 5612 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0b61c341374ced8690903b04a31804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:29 UTC) from {username='slave'} at 127.0.0.1:59862
I20260812 03:56:29.311326 5759 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0b61c341374ced8690903b04a31804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:29 UTC) from {username='slave'} at 127.0.0.1:42718
I20260812 03:56:29.311537 5467 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0b61c341374ced8690903b04a31804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:29 UTC) from {username='slave'} at 127.0.0.1:42882
I20260812 03:56:29.311581 5871 tablet_replica.cc:330] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: stopping tablet replica
I20260812 03:56:29.311681 5871 raft_consensus.cc:2229] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:29.311687 5872 tablet_replica.cc:330] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: stopping tablet replica
I20260812 03:56:29.311784 5871 raft_consensus.cc:2258] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:29.311789 5872 raft_consensus.cc:2229] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:29.311851 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 5394
I20260812 03:56:29.311883 5872 raft_consensus.cc:2258] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:29.314370 5872 ts_tablet_manager.cc:1905] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:29.315106 5871 ts_tablet_manager.cc:1905] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:29.317075 5872 ts_tablet_manager.cc:1918] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:29.317106 5871 ts_tablet_manager.cc:1918] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:29.317166 5872 log.cc:1196] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/eb0b61c341374ced8690903b04a31804
I20260812 03:56:29.317181 5871 log.cc:1196] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/eb0b61c341374ced8690903b04a31804
I20260812 03:56:29.317456 5872 ts_tablet_manager.cc:1939] T eb0b61c341374ced8690903b04a31804 P 03aee4c35c2c4428a6818b612e0fd5ee: Deleting consensus metadata
I20260812 03:56:29.317502 5871 ts_tablet_manager.cc:1939] T eb0b61c341374ced8690903b04a31804 P 22876415e0664a098fa9ecd1ad3c5238: Deleting consensus metadata
I20260812 03:56:29.318279 5312 catalog_manager.cc:4841] TS 03aee4c35c2c4428a6818b612e0fd5ee (127.17.54.195:37133): tablet eb0b61c341374ced8690903b04a31804 (table test-table [id=df4d4d0312624eeaa7c21071045e3dee]) successfully deleted
I20260812 03:56:29.318535 5314 catalog_manager.cc:4841] TS 22876415e0664a098fa9ecd1ad3c5238 (127.17.54.194:43127): tablet eb0b61c341374ced8690903b04a31804 (table test-table [id=df4d4d0312624eeaa7c21071045e3dee]) successfully deleted
W20260812 03:56:29.321722 5312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:33851 (error 108)
I20260812 03:56:29.321965 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 5541
I20260812 03:56:29.323210 5842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:29.322114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33851 (local address 127.0.0.1:42896)
0812 03:56:29.323108 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:29.323163 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":965}
W20260812 03:56:29.324540 5312 catalog_manager.cc:4584] TS 43c36e0e70ba4f8f95e5833d746d08a5 (127.17.54.193:33851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb0b61c341374ced8690903b04a31804: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33851: connect: Connection refused (error 111)
I20260812 03:56:29.330109 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 5688
I20260812 03:56:29.337642 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 5295
2026-08-12T03:56:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (823 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:29 dist-test-slave-tm1g krb5kdc[5878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:29 dist-test-slave-tm1g krb5kdc[5878](info): set up 2 sockets
Aug 12 03:56:29 dist-test-slave-tm1g krb5kdc[5878](info): commencing operation
krb5kdc: starting...
W20260812 03:56:30.477396 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:30 dist-test-slave-tm1g krb5kdc[5878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:56:30.606274 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:36401
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:36401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:30.698900 5893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:30.699175 5893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:30.699218 5893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:30.700582 5893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:30.700667 5893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:30.700699 5893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:30.700766 5893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:30.700803 5893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:30.702512 5893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:36401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:36401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:30.702814 5893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:30.703068 5893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:30.708478 5899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:30.708559 5898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:30.708498 5902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:30.708537 5893 server_base.cc:1034] running on GCE node
I20260812 03:56:30.708958 5893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:30.709304 5893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:30.710476 5893 hybrid_clock.cc:648] HybridClock initialized: now 1786506990710456 us; error 41 us; skew 500 ppm
Aug 12 03:56:30 dist-test-slave-tm1g krb5kdc[5878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:30.713109 5893 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:30.713513 5893 webserver.cc:460] Webserver started at http://127.17.54.254:34533/ using document root <none> and password file <none>
I20260812 03:56:30.713696 5893 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:30.713743 5893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:30.713835 5893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:30.714967 5893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "ef2f369d6bae4c4c93cda84fe738ddf1"
format_stamp: "Formatted at 2026-08-12 03:56:30 on dist-test-slave-tm1g"
server_key: "65ec7251cd1dac6c71359cd093a572ff"
server_key_iv: "3e1786b1a8badfabccb6301ab1313e93"
server_key_version: "encryptionkey@0"
I20260812 03:56:30.715271 5893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "ef2f369d6bae4c4c93cda84fe738ddf1"
format_stamp: "Formatted at 2026-08-12 03:56:30 on dist-test-slave-tm1g"
server_key: "65ec7251cd1dac6c71359cd093a572ff"
server_key_iv: "3e1786b1a8badfabccb6301ab1313e93"
server_key_version: "encryptionkey@0"
I20260812 03:56:30.716588 5893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:56:30.717309 5909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:30.717536 5893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:30.717634 5893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "ef2f369d6bae4c4c93cda84fe738ddf1"
format_stamp: "Formatted at 2026-08-12 03:56:30 on dist-test-slave-tm1g"
server_key: "65ec7251cd1dac6c71359cd093a572ff"
server_key_iv: "3e1786b1a8badfabccb6301ab1313e93"
server_key_version: "encryptionkey@0"
I20260812 03:56:30.717697 5893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:30.737226 5893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:30.739892 5893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:30.740023 5893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:30.744396 5893 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:36401
I20260812 03:56:30.744516 5969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:36401 every 8 connection(s)
I20260812 03:56:30.744846 5893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:30.745824 5970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:30.748227 5970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1: Bootstrap starting.
I20260812 03:56:30.748852 5970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:30.749168 5970 log.cc:824] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:30.749904 5970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1: No bootstrap required, opened a new log
I20260812 03:56:30.751616 5970 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f369d6bae4c4c93cda84fe738ddf1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36401 } }
I20260812 03:56:30.751780 5970 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:30.751827 5970 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2f369d6bae4c4c93cda84fe738ddf1, State: Initialized, Role: FOLLOWER
I20260812 03:56:30.752000 5970 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f369d6bae4c4c93cda84fe738ddf1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36401 } }
I20260812 03:56:30.752086 5970 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:30.752133 5970 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:30.752185 5970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:30.752485 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 5893
I20260812 03:56:30.752606 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:30.752807 17627 external_mini_cluster.cc:1428] Setting key 4fc6587be73786465b1fb6fab98f58d5
I20260812 03:56:30.752916 5970 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f369d6bae4c4c93cda84fe738ddf1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36401 } }
I20260812 03:56:30.753050 5970 leader_election.cc:302] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef2f369d6bae4c4c93cda84fe738ddf1; no voters:
I20260812 03:56:30.753265 5970 leader_election.cc:288] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:30.753324 5973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:30.753485 5973 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [term 1 LEADER]: Becoming Leader. State: Replica: ef2f369d6bae4c4c93cda84fe738ddf1, State: Running, Role: LEADER
I20260812 03:56:30.753548 5970 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:30.753592 5973 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f369d6bae4c4c93cda84fe738ddf1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36401 } }
I20260812 03:56:30.753938 5973 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef2f369d6bae4c4c93cda84fe738ddf1. Latest consensus state: current_term: 1 leader_uuid: "ef2f369d6bae4c4c93cda84fe738ddf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f369d6bae4c4c93cda84fe738ddf1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36401 } } }
I20260812 03:56:30.754026 5973 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:30.754223 5977 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef2f369d6bae4c4c93cda84fe738ddf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f369d6bae4c4c93cda84fe738ddf1" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 36401 } } }
I20260812 03:56:30.754379 5977 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:30.754855 5982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:30.755396 5982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:30.758745 5982 catalog_manager.cc:1347] Generated new cluster ID: 51f501128b2c45a5855a53913013b020
I20260812 03:56:30.758823 5982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:56:30 dist-test-slave-tm1g krb5kdc[5878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:30.765754 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:30.753334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.17.54.254:36401)
0812 03:56:30.753506 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:30.753517 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:30.754483 (+ 966us) server_negotiation.cc:407] Connection header received
0812 03:56:30.754608 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:30.754614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:30.754682 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:30.754822 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:30.755726 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:30.757115 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:30.757915 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:30.758609 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:30.758719 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:30.761693 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:30.761698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:30.761698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:30.761713 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:30.763882 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:30.764282 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:30.764286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:30.764286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:30.764415 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:30.764617 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:30.764619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:30.764619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:30.764971 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:30.765127 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:30.765228 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:56:30.765512 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260812 03:56:30.785321 5982 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:30.786114 5982 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:30.790083 5982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1: Generated new TSK 0
I20260812 03:56:30.790279 5982 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:56:30.821866 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36401
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:30.906361 5997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:30.906579 5997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:30.906603 5997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:30.907841 5997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:30.907884 5997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:30.907912 5997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:56:30.909375 5997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:30.909643 5997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:30.909883 5997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:30.915807 6002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:30.915860 5997 server_base.cc:1034] running on GCE node
W20260812 03:56:30.915827 6003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:30.915815 6005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:30.916263 5997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:30.916544 5997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:30.917739 5997 hybrid_clock.cc:648] HybridClock initialized: now 1786506990917720 us; error 48 us; skew 500 ppm
Aug 12 03:56:30 dist-test-slave-tm1g krb5kdc[5878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:30.920579 5997 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:56:30.921057 5997 webserver.cc:460] Webserver started at http://127.17.54.193:33083/ using document root <none> and password file <none>
I20260812 03:56:30.921259 5997 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:30.921301 5997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:30.921401 5997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:30.922554 5997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "ed3c358aaa5c462580ace29fe116d34a"
format_stamp: "Formatted at 2026-08-12 03:56:30 on dist-test-slave-tm1g"
server_key: "d5a9d3a53a23c90a368fb2fd25eaea40"
server_key_iv: "daec12393a0bd30d8f2b8504bda7a042"
server_key_version: "encryptionkey@0"
I20260812 03:56:30.922870 5997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "ed3c358aaa5c462580ace29fe116d34a"
format_stamp: "Formatted at 2026-08-12 03:56:30 on dist-test-slave-tm1g"
server_key: "d5a9d3a53a23c90a368fb2fd25eaea40"
server_key_iv: "daec12393a0bd30d8f2b8504bda7a042"
server_key_version: "encryptionkey@0"
I20260812 03:56:30.924248 5997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:56:30.925093 6013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:30.925323 5997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:30.925421 5997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "ed3c358aaa5c462580ace29fe116d34a"
format_stamp: "Formatted at 2026-08-12 03:56:30 on dist-test-slave-tm1g"
server_key: "d5a9d3a53a23c90a368fb2fd25eaea40"
server_key_iv: "daec12393a0bd30d8f2b8504bda7a042"
server_key_version: "encryptionkey@0"
I20260812 03:56:30.925483 5997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:30.955740 5997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:30.957943 5997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:30.958067 5997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:30.958269 5997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:30.958634 5997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:30.958678 5997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:30.958726 5997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:30.958752 5997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:30.964926 5997 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:43751
I20260812 03:56:30.964994 6138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:43751 every 8 connection(s)
I20260812 03:56:30.965371 5997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:56:30.968014 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 5997
I20260812 03:56:30.968139 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:56:30.968356 17627 external_mini_cluster.cc:1428] Setting key ff83f98f1009e3201ca598d70fc0c06a
Aug 12 03:56:30 dist-test-slave-tm1g krb5kdc[5878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:30.976528 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:30.966239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:56021 (local address 127.17.54.254:36401)
0812 03:56:30.966354 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:30.966356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:30.966566 (+ 210us) server_negotiation.cc:407] Connection header received
0812 03:56:30.967103 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:30.967104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:30.967122 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:30.967179 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:30.968173 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:30.968996 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:30.970595 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:30.970808 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:30.970871 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:30.973196 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:30.973198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:30.973199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:30.973207 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:30.974925 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:30.975505 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:30.975506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:30.975507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:30.975561 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:30.975856 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:30.975857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:30.975857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:30.975961 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:30.976011 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:30.976370 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 03:56:30.976444 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":44}
I20260812 03:56:30.976820 6141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:30.966198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36401 (local address 127.17.54.193:56021)
0812 03:56:30.966433 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:30.966472 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:30.966991 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:30.967251 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:30.967260 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:30.967837 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:30.968033 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:30.968038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:30.969106 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:30.969107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:30.970512 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:30.970515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:30.970903 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:30.970904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:30.971068 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:30.971679 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:30.971693 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:30.973095 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:30.975019 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:30.975029 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:30.975032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:30.975405 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:30.975633 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:30.975634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:30.975635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:30.975785 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:30.976031 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:30.976040 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:30.976217 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 03:56:30.976357 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 03:56:30.976548 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":128,"threads_started":1}
I20260812 03:56:30.977396 6139 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36401
I20260812 03:56:30.977545 6139 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:30.977747 6139 heartbeater.cc:511] Master 127.17.54.254:36401 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
I20260812 03:56:30.978354 5930 ts_manager.cc:194] Registered new tserver with Master: ed3c358aaa5c462580ace29fe116d34a (127.17.54.193:43751)
I20260812 03:56:30.978989 5930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:56021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:56:30.997987 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36401
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:31.081693 6145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:31.081964 6145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:31.082005 6145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:31.083222 6145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:31.083300 6145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:31.083346 6145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:56:31.084880 6145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:31.085173 6145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:31.085443 6145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:31.090792 6153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:31.090812 6151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:31.090826 6145 server_base.cc:1034] running on GCE node
W20260812 03:56:31.090792 6150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:31.091312 6145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:31.091572 6145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:31.092752 6145 hybrid_clock.cc:648] HybridClock initialized: now 1786506991092731 us; error 37 us; skew 500 ppm
Aug 12 03:56:31 dist-test-slave-tm1g krb5kdc[5878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506991, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:31.095639 6145 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:56:31.096187 6145 webserver.cc:460] Webserver started at http://127.17.54.194:42727/ using document root <none> and password file <none>
I20260812 03:56:31.096455 6145 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:31.096503 6145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:31.096660 6145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:31.097744 6145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "ea173f58ab0c4540acbbd986177cd7c3"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "6cadee7e02621e60533084975b5c3c0f"
server_key_iv: "b5475ab7a114251b147e61f9c683b6b7"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.098122 6145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "ea173f58ab0c4540acbbd986177cd7c3"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "6cadee7e02621e60533084975b5c3c0f"
server_key_iv: "b5475ab7a114251b147e61f9c683b6b7"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.099586 6145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:31.100476 6160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.100754 6145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:31.100890 6145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "ea173f58ab0c4540acbbd986177cd7c3"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "6cadee7e02621e60533084975b5c3c0f"
server_key_iv: "b5475ab7a114251b147e61f9c683b6b7"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.100991 6145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:31.117152 6145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:31.119601 6145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:31.119773 6145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:31.120034 6145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:31.120361 6145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:31.120416 6145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.120463 6145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:31.120493 6145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.126803 6145 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:37739
I20260812 03:56:31.126858 6288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:37739 every 8 connection(s)
I20260812 03:56:31.127171 6145 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:56:31 dist-test-slave-tm1g krb5kdc[5878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506991, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:31.134044 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 6145
I20260812 03:56:31.134187 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:56:31.134459 17627 external_mini_cluster.cc:1428] Setting key 4687c4542848344a791aaebd71761625
I20260812 03:56:31.137518 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.127975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:49949 (local address 127.17.54.254:36401)
0812 03:56:31.128055 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.128058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.128229 (+ 171us) server_negotiation.cc:407] Connection header received
0812 03:56:31.128776 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.128777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.128790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.128865 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:31.129913 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.130521 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.131522 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.131740 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.131827 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.134103 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:31.134104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:31.134105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:31.134113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:31.135856 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:31.136304 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:31.136306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:31.136306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:31.136382 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:31.136681 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:31.136682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:31.136682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:31.136806 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:31.136896 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.137184 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.137279 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:56:31.137749 6291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.127884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36401 (local address 127.17.54.194:49949)
0812 03:56:31.128085 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.128130 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:31.128664 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:31.128978 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:31.128982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:31.129612 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:31.129813 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.129817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.130617 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.130618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.131427 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.131429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.131839 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.131840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.131974 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.132502 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:31.132510 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:31.133987 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:31.135946 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:31.135954 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:31.135956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:31.136188 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:31.136484 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:31.136485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:31.136485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:31.136603 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:31.136885 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:31.136893 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:31.137046 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 03:56:31.137162 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 03:56:31.137461 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260812 03:56:31.138538 6289 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36401
I20260812 03:56:31.138630 6289 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:31.138862 6289 heartbeater.cc:511] Master 127.17.54.254:36401 requested a full tablet report, sending...
I20260812 03:56:31.139304 5930 ts_manager.cc:194] Registered new tserver with Master: ea173f58ab0c4540acbbd986177cd7c3 (127.17.54.194:37739)
I20260812 03:56:31.139860 5930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:49949
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:56:31.165066 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36401
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:31.249882 6295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:31.250111 6295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:31.250136 6295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:31.251392 6295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:31.251439 6295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:31.251467 6295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:56:31.253043 6295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:31.253290 6295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:31.253544 6295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:31.259248 6303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:31.259267 6301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:31.259263 6300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:31.259498 6295 server_base.cc:1034] running on GCE node
I20260812 03:56:31.259728 6295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:31.259941 6295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:31.261144 6295 hybrid_clock.cc:648] HybridClock initialized: now 1786506991261102 us; error 32 us; skew 500 ppm
Aug 12 03:56:31 dist-test-slave-tm1g krb5kdc[5878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506991, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:31.263924 6295 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:56:31.264406 6295 webserver.cc:460] Webserver started at http://127.17.54.195:36259/ using document root <none> and password file <none>
I20260812 03:56:31.264653 6295 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:31.264700 6295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:31.264842 6295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:31.265992 6295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "93cc06c461974f1486db0802cc153850"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "0282850a0a74601092d12cd30294f9d3"
server_key_iv: "e8cbd1cd5b9f14cdd114b6a9e4735570"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.266394 6295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "93cc06c461974f1486db0802cc153850"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "0282850a0a74601092d12cd30294f9d3"
server_key_iv: "e8cbd1cd5b9f14cdd114b6a9e4735570"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.267786 6295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:31.268564 6311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.268784 6295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:31.268908 6295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "93cc06c461974f1486db0802cc153850"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "0282850a0a74601092d12cd30294f9d3"
server_key_iv: "e8cbd1cd5b9f14cdd114b6a9e4735570"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.268988 6295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:31.283524 6295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:31.285912 6295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:31.286099 6295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:31.286423 6295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:31.286778 6295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:31.286846 6295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.286883 6295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:31.286917 6295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.293260 6295 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:36373
I20260812 03:56:31.293490 6434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:36373 every 8 connection(s)
I20260812 03:56:31.293674 6295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:56:31 dist-test-slave-tm1g krb5kdc[5878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506991, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:31.301051 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 6295
I20260812 03:56:31.301132 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:56:31.301318 17627 external_mini_cluster.cc:1428] Setting key 28a8af20205e4a3ab8fb06f928bed3f9
I20260812 03:56:31.304260 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.294479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:52595 (local address 127.17.54.254:36401)
0812 03:56:31.294531 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.294534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.294792 (+ 258us) server_negotiation.cc:407] Connection header received
0812 03:56:31.295270 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.295271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.295281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.295333 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:31.296259 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.296754 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.297733 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.297918 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.297975 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.300462 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:31.300464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:31.300464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:31.300472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:31.302424 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:31.303046 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:31.303048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:31.303049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:31.303130 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:31.303512 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:31.303513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:31.303513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:31.303678 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:31.303777 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.304121 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.304193 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:56:31.304561 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.294487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:36401 (local address 127.17.54.195:52595)
0812 03:56:31.294644 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.294682 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:31.295180 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:31.295396 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:31.295399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:31.295983 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:31.296185 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.296190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.296828 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.296828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.297648 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.297650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.298031 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.298032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.298157 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.298784 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:31.298793 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:31.300284 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:31.302499 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:31.302507 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:31.302509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:31.302944 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:31.303215 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:31.303216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:31.303216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:31.303423 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:31.303767 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:31.303775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:31.303956 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 03:56:31.304079 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 03:56:31.304246 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260812 03:56:31.305083 6435 heartbeater.cc:347] Connected to a master server at 127.17.54.254:36401
I20260812 03:56:31.305202 6435 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:31.305414 6435 heartbeater.cc:511] Master 127.17.54.254:36401 requested a full tablet report, sending...
I20260812 03:56:31.305866 5930 ts_manager.cc:194] Registered new tserver with Master: 93cc06c461974f1486db0802cc153850 (127.17.54.195:36373)
I20260812 03:56:31.306347 5930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:52595
I20260812 03:56:31.313520 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:56:31.326748 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.315054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.17.54.254:36401)
0812 03:56:31.315124 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.315126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.315220 (+ 94us) server_negotiation.cc:407] Connection header received
0812 03:56:31.315284 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.315285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.315293 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.315347 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:31.315919 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.316426 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.317176 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.317560 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.317635 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.319041 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:31.319042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:31.319042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:31.319049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:31.325314 (+ 6265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:31.325821 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:31.325822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:31.325822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:31.325888 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:31.326131 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:31.326132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:31.326132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:31.326278 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:31.326422 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.326623 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.326679 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:56:31.328999 5930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:56:31.330873 5930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:56:31.337975 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.333997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.17.54.194:37739)
0812 03:56:31.334180 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.334184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.334193 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:56:31.334225 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.334227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.334239 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.334339 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:31.334773 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.335500 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.336730 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.337485 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.337534 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.337597 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.337842 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.337886 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260812 03:56:31.337960 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.333682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:37739 (local address 127.0.0.1:57258)
0812 03:56:31.333954 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.333964 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:31.334008 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:31.334384 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:31.334388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:31.334390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:31.334478 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.334480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.335609 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.335610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.336624 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.336627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.337656 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.337656 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.337729 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.337739 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 03:56:31.337821 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:56:31.337880 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":117,"threads_started":1}
I20260812 03:56:31.338883 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.333682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:43751 (local address 127.0.0.1:38636)
0812 03:56:31.334191 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.334202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:31.334252 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:31.334654 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:31.334654 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:31.334655 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:31.334724 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.334726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.336257 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.336258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.337309 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.337312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.338626 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.338627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.338691 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.338697 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:56:31.338758 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:56:31.338805 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":7,"thread_start_us":120,"threads_started":1}
I20260812 03:56:31.338797 6218 tablet_service.cc:1467] Processing CreateTablet for tablet 142d9687b7c3421382d31c9a1f6cc799 (DEFAULT_TABLE table=test-table [id=f60c5af7b613454ca6b2cc85dd588df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:31.339032 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.333674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:36373 (local address 127.0.0.1:36694)
0812 03:56:31.333841 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.333884 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:31.333959 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:31.334650 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:31.334651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:31.334653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:31.334715 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.334717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.336245 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.336246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.337536 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.337539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.338780 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.338781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.338849 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.338854 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:56:31.338890 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 03:56:31.338937 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":50,"threads_started":1}
I20260812 03:56:31.339221 6218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 142d9687b7c3421382d31c9a1f6cc799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:31.339395 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.333984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.17.54.193:43751)
0812 03:56:31.334496 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.334501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.334510 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:56:31.334551 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.334553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.334568 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.334649 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:31.334827 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.336135 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.337446 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.338526 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.338615 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.338679 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.338774 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.338823 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":100,"threads_started":1}
I20260812 03:56:31.339627 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.333766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.17.54.195:36373)
0812 03:56:31.334445 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.334450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.334463 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:56:31.334509 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.334511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.334530 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.334802 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:31.334900 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.336103 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.337628 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.338653 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.339332 (+ 679us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.339425 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.339458 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.339519 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":99,"threads_started":1}
I20260812 03:56:31.340101 6066 tablet_service.cc:1467] Processing CreateTablet for tablet 142d9687b7c3421382d31c9a1f6cc799 (DEFAULT_TABLE table=test-table [id=f60c5af7b613454ca6b2cc85dd588df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:31.340446 6066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 142d9687b7c3421382d31c9a1f6cc799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:31.342913 6453 tablet_bootstrap.cc:492] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Bootstrap starting.
I20260812 03:56:31.342883 6363 tablet_service.cc:1467] Processing CreateTablet for tablet 142d9687b7c3421382d31c9a1f6cc799 (DEFAULT_TABLE table=test-table [id=f60c5af7b613454ca6b2cc85dd588df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:31.343305 6363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 142d9687b7c3421382d31c9a1f6cc799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:31.343784 6453 tablet_bootstrap.cc:654] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:31.343906 6454 tablet_bootstrap.cc:492] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a: Bootstrap starting.
I20260812 03:56:31.344488 6454 tablet_bootstrap.cc:654] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:31.344558 6453 log.cc:824] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:31.345144 6454 log.cc:824] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:31.345527 6453 tablet_bootstrap.cc:492] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: No bootstrap required, opened a new log
I20260812 03:56:31.345610 6453 ts_tablet_manager.cc:1397] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:56:31.346020 6454 tablet_bootstrap.cc:492] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a: No bootstrap required, opened a new log
I20260812 03:56:31.346115 6454 ts_tablet_manager.cc:1397] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:56:31.346966 6457 tablet_bootstrap.cc:492] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: Bootstrap starting.
I20260812 03:56:31.347780 6457 tablet_bootstrap.cc:654] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:31.347707 6453 raft_consensus.cc:348] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.347849 6453 raft_consensus.cc:374] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:31.347941 6453 raft_consensus.cc:729] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea173f58ab0c4540acbbd986177cd7c3, State: Initialized, Role: FOLLOWER
I20260812 03:56:31.348071 6453 consensus_queue.cc:260] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.348421 6453 ts_tablet_manager.cc:1428] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:56:31.348479 6457 log.cc:824] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:31.348448 6454 raft_consensus.cc:348] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.348591 6289 heartbeater.cc:503] Master 127.17.54.254:36401 was elected leader, sending a full tablet report...
I20260812 03:56:31.348623 6454 raft_consensus.cc:374] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:31.348675 6454 raft_consensus.cc:729] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed3c358aaa5c462580ace29fe116d34a, State: Initialized, Role: FOLLOWER
I20260812 03:56:31.348825 6454 consensus_queue.cc:260] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.349167 6139 heartbeater.cc:503] Master 127.17.54.254:36401 was elected leader, sending a full tablet report...
I20260812 03:56:31.349149 6454 ts_tablet_manager.cc:1428] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:56:31.349589 6457 tablet_bootstrap.cc:492] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: No bootstrap required, opened a new log
I20260812 03:56:31.349663 6457 ts_tablet_manager.cc:1397] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:56:31.351472 6457 raft_consensus.cc:348] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.351610 6457 raft_consensus.cc:374] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:31.351634 6457 raft_consensus.cc:729] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93cc06c461974f1486db0802cc153850, State: Initialized, Role: FOLLOWER
I20260812 03:56:31.351751 6457 consensus_queue.cc:260] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.351998 6457 ts_tablet_manager.cc:1428] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:56:31.352067 6435 heartbeater.cc:503] Master 127.17.54.254:36401 was elected leader, sending a full tablet report...
W20260812 03:56:31.377828 6290 tablet.cc:2369] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:56:31.466233 6140 tablet.cc:2369] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:56:31.467693 6458 raft_consensus.cc:482] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:31.467814 6458 raft_consensus.cc:504] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.468406 6458 leader_election.cc:288] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed3c358aaa5c462580ace29fe116d34a (127.17.54.193:43751), 93cc06c461974f1486db0802cc153850 (127.17.54.195:36373)
I20260812 03:56:31.472653 6291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.468594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:43751 (local address 127.17.54.194:38739)
0812 03:56:31.468657 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.468669 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:31.468717 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:31.468949 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:31.468950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:31.468952 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:31.469065 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.469067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.470045 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.470046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.471467 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.471468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.472299 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.472300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.472459 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.472462 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:31.472519 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:56:31.472553 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":128}
I20260812 03:56:31.472661 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.468594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:36373 (local address 127.17.54.194:41553)
0812 03:56:31.468831 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.468841 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:31.468893 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:31.469055 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:31.469055 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:31.469057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:31.469118 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.469120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.470045 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.470046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.471456 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.471459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.472299 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.472300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.472412 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.472417 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:56:31.472491 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:56:31.472544 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":72,"thread_start_us":127,"threads_started":1}
I20260812 03:56:31.472690 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.468671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:38739 (local address 127.17.54.193:43751)
0812 03:56:31.468733 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.468735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.468793 (+ 58us) server_negotiation.cc:407] Connection header received
0812 03:56:31.468819 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.468820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.468830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.468933 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:31.469166 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.469941 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.471577 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.472182 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.472267 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.472277 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.472564 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.472622 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:56:31.472707 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.468823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:41553 (local address 127.17.54.195:36373)
0812 03:56:31.468900 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.468902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.468913 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:56:31.468969 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.468970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.468979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.469045 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:31.469191 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.469934 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.471587 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.472176 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.472267 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.472277 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.472598 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.472646 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:56:31.473097 6385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142d9687b7c3421382d31c9a1f6cc799" candidate_uuid: "ea173f58ab0c4540acbbd986177cd7c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93cc06c461974f1486db0802cc153850" is_pre_election: true
I20260812 03:56:31.473096 6090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142d9687b7c3421382d31c9a1f6cc799" candidate_uuid: "ea173f58ab0c4540acbbd986177cd7c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed3c358aaa5c462580ace29fe116d34a" is_pre_election: true
I20260812 03:56:31.473273 6090 raft_consensus.cc:2454] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea173f58ab0c4540acbbd986177cd7c3 in term 0.
I20260812 03:56:31.473273 6385 raft_consensus.cc:2454] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea173f58ab0c4540acbbd986177cd7c3 in term 0.
I20260812 03:56:31.473541 6162 leader_election.cc:302] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93cc06c461974f1486db0802cc153850, ea173f58ab0c4540acbbd986177cd7c3; no voters:
I20260812 03:56:31.473685 6458 raft_consensus.cc:2781] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:56:31.473744 6458 raft_consensus.cc:482] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:56:31.473769 6458 raft_consensus.cc:3037] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:31.474555 6458 raft_consensus.cc:504] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.474710 6458 leader_election.cc:288] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [CANDIDATE]: Term 1 election: Requested vote from peers ed3c358aaa5c462580ace29fe116d34a (127.17.54.193:43751), 93cc06c461974f1486db0802cc153850 (127.17.54.195:36373)
I20260812 03:56:31.474891 6090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142d9687b7c3421382d31c9a1f6cc799" candidate_uuid: "ea173f58ab0c4540acbbd986177cd7c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed3c358aaa5c462580ace29fe116d34a"
I20260812 03:56:31.474946 6385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142d9687b7c3421382d31c9a1f6cc799" candidate_uuid: "ea173f58ab0c4540acbbd986177cd7c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93cc06c461974f1486db0802cc153850"
I20260812 03:56:31.474975 6090 raft_consensus.cc:3037] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:31.475051 6385 raft_consensus.cc:3037] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:31.475732 6090 raft_consensus.cc:2454] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea173f58ab0c4540acbbd986177cd7c3 in term 1.
I20260812 03:56:31.475942 6163 leader_election.cc:302] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea173f58ab0c4540acbbd986177cd7c3, ed3c358aaa5c462580ace29fe116d34a; no voters:
I20260812 03:56:31.475944 6385 raft_consensus.cc:2454] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea173f58ab0c4540acbbd986177cd7c3 in term 1.
I20260812 03:56:31.476044 6458 raft_consensus.cc:2781] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:31.476231 6458 raft_consensus.cc:686] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 1 LEADER]: Becoming Leader. State: Replica: ea173f58ab0c4540acbbd986177cd7c3, State: Running, Role: LEADER
I20260812 03:56:31.476359 6458 consensus_queue.cc:237] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } }
I20260812 03:56:31.477270 5930 catalog_manager.cc:5495] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea173f58ab0c4540acbbd986177cd7c3 (127.17.54.194). New cstate: current_term: 1 leader_uuid: "ea173f58ab0c4540acbbd986177cd7c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea173f58ab0c4540acbbd986177cd7c3" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 37739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 } health_report { overall_health: UNKNOWN } } }
I20260812 03:56:31.485592 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.483069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57266 (local address 127.17.54.194:37739)
0812 03:56:31.483142 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.483144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.483153 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:56:31.483190 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.483191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.483211 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.483302 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:56:31.483478 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.483954 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.484724 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.484956 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.485036 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.485197 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:56:31.485307 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:56:31.485420 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.485476 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.485487 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:56:31.487560 6385 raft_consensus.cc:1261] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 1 FOLLOWER]: Refusing update from remote peer ea173f58ab0c4540acbbd986177cd7c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:31.487560 6090 raft_consensus.cc:1261] T 142d9687b7c3421382d31c9a1f6cc799 P ed3c358aaa5c462580ace29fe116d34a [term 1 FOLLOWER]: Refusing update from remote peer ea173f58ab0c4540acbbd986177cd7c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:31.487973 6464 consensus_queue.cc:1035] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed3c358aaa5c462580ace29fe116d34a" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 43751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:31.488116 6458 consensus_queue.cc:1035] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93cc06c461974f1486db0802cc153850" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 36373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:31.490739 6469 mvcc.cc:205] Tried to move back new op lower bound from 7317532637129945088 to 7317532637087662080. Current Snapshot: MvccSnapshot[applied={T|T < 7317532637129945088}]
I20260812 03:56:31.492002 6468 mvcc.cc:205] Tried to move back new op lower bound from 7317532637129945088 to 7317532637087662080. Current Snapshot: MvccSnapshot[applied={T|T < 7317532637129945088 or (T in {7317532637129945088})}]
I20260812 03:56:31.493669 5930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:31.493774 5930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:31.494588 5930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef2f369d6bae4c4c93cda84fe738ddf1: Sending DeleteTablet for 3 replicas of tablet 142d9687b7c3421382d31c9a1f6cc799
I20260812 03:56:31.494980 6218 tablet_service.cc:1514] Processing DeleteTablet for tablet 142d9687b7c3421382d31c9a1f6cc799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:31 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:57258
I20260812 03:56:31.495172 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 5997
I20260812 03:56:31.495208 6066 tablet_service.cc:1514] Processing DeleteTablet for tablet 142d9687b7c3421382d31c9a1f6cc799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:31 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:38636
I20260812 03:56:31.499063 6363 tablet_service.cc:1514] Processing DeleteTablet for tablet 142d9687b7c3421382d31c9a1f6cc799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:31 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:36694
I20260812 03:56:31.501276 6480 tablet_replica.cc:330] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: stopping tablet replica
I20260812 03:56:31.501422 6480 raft_consensus.cc:2229] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:31.501554 6480 raft_consensus.cc:2258] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:31.501816 6478 tablet_replica.cc:330] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: stopping tablet replica
I20260812 03:56:31.501919 6478 raft_consensus.cc:2229] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:56:31.502043 6478 raft_consensus.cc:2258] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:31.513015 6478 ts_tablet_manager.cc:1905] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:31.515892 6478 ts_tablet_manager.cc:1918] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:31.516028 6478 log.cc:1196] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/142d9687b7c3421382d31c9a1f6cc799
I20260812 03:56:31.516342 6478 ts_tablet_manager.cc:1939] T 142d9687b7c3421382d31c9a1f6cc799 P ea173f58ab0c4540acbbd986177cd7c3: Deleting consensus metadata
I20260812 03:56:31.516837 5913 catalog_manager.cc:4841] TS ea173f58ab0c4540acbbd986177cd7c3 (127.17.54.194:37739): tablet 142d9687b7c3421382d31c9a1f6cc799 (table test-table [id=f60c5af7b613454ca6b2cc85dd588df1]) successfully deleted
W20260812 03:56:31.517777 5912 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:43751 (error 108)
I20260812 03:56:31.518424 6480 ts_tablet_manager.cc:1905] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:31.518641 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.518403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:43751 (local address 127.0.0.1:38652)
0812 03:56:31.518545 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.518601 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:43751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260812 03:56:31.518765 5912 catalog_manager.cc:4584] TS ed3c358aaa5c462580ace29fe116d34a (127.17.54.193:43751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 142d9687b7c3421382d31c9a1f6cc799: Network error: Client connection negotiation failed: client connection to 127.17.54.193:43751: connect: Connection refused (error 111)
I20260812 03:56:31.519029 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 6145
I20260812 03:56:31.522778 6480 ts_tablet_manager.cc:1918] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:31.522918 6480 log.cc:1196] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/142d9687b7c3421382d31c9a1f6cc799
I20260812 03:56:31.523190 6480 ts_tablet_manager.cc:1939] T 142d9687b7c3421382d31c9a1f6cc799 P 93cc06c461974f1486db0802cc153850: Deleting consensus metadata
I20260812 03:56:31.523762 5911 catalog_manager.cc:4841] TS 93cc06c461974f1486db0802cc153850 (127.17.54.195:36373): tablet 142d9687b7c3421382d31c9a1f6cc799 (table test-table [id=f60c5af7b613454ca6b2cc85dd588df1]) successfully deleted
I20260812 03:56:31.527993 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 6295
I20260812 03:56:31.536434 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 5893
2026-08-12T03:56:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2199 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T03:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:31Z Disabled control of system clock
I20260812 03:56:31.569640 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37517
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:37517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:56:31.657217 6485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:31.657497 6485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:31.657536 6485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:31.658805 6485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:31.658874 6485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:31.658896 6485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:31.658931 6485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:31.658979 6485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:31.660542 6485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:37517
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:31.660866 6485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:31.661203 6485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:31.666383 6495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:31.666352 6492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:31.666435 6493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:31.667038 6485 server_base.cc:1034] running on GCE node
I20260812 03:56:31.667307 6485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:31.667632 6485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:31.668838 6485 hybrid_clock.cc:648] HybridClock initialized: now 1786506991668810 us; error 43 us; skew 500 ppm
I20260812 03:56:31.670507 6485 webserver.cc:460] Webserver started at http://127.17.54.254:44577/ using document root <none> and password file <none>
I20260812 03:56:31.670822 6485 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:31.670910 6485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:31.671087 6485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:31.672331 6485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "ac648cca89d043d1a4bf656d800467b3"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "1fae8484bf68752adeb0b910ab0008d4"
server_key_iv: "7039358b67ddf7a06011e978dbbdddb0"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.672734 6485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "ac648cca89d043d1a4bf656d800467b3"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "1fae8484bf68752adeb0b910ab0008d4"
server_key_iv: "7039358b67ddf7a06011e978dbbdddb0"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.674558 6485 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 03:56:31.675566 6502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.675761 6485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:31.675884 6485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "ac648cca89d043d1a4bf656d800467b3"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "1fae8484bf68752adeb0b910ab0008d4"
server_key_iv: "7039358b67ddf7a06011e978dbbdddb0"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.675990 6485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:31.697901 6485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:31.698431 6485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:31.698622 6485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:31.704627 6485 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:37517
I20260812 03:56:31.704654 6559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:37517 every 8 connection(s)
I20260812 03:56:31.705183 6485 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:31.706291 6560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:31.708702 6560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3: Bootstrap starting.
I20260812 03:56:31.709316 6560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:31.709636 6560 log.cc:824] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:31.710358 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 6485
I20260812 03:56:31.710484 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:31.710554 6560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3: No bootstrap required, opened a new log
I20260812 03:56:31.710709 17627 external_mini_cluster.cc:1428] Setting key 3584aeae95425f00f49a933a812a22fe
I20260812 03:56:31.713322 6560 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac648cca89d043d1a4bf656d800467b3" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37517 } }
I20260812 03:56:31.713495 6560 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:31.713537 6560 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac648cca89d043d1a4bf656d800467b3, State: Initialized, Role: FOLLOWER
I20260812 03:56:31.713675 6560 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac648cca89d043d1a4bf656d800467b3" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37517 } }
I20260812 03:56:31.713757 6560 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:31.713786 6560 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:31.713832 6560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:31.714620 6560 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac648cca89d043d1a4bf656d800467b3" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37517 } }
I20260812 03:56:31.714716 6560 leader_election.cc:302] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac648cca89d043d1a4bf656d800467b3; no voters:
I20260812 03:56:31.714962 6560 leader_election.cc:288] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:31.714985 6566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:31.715152 6566 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [term 1 LEADER]: Becoming Leader. State: Replica: ac648cca89d043d1a4bf656d800467b3, State: Running, Role: LEADER
I20260812 03:56:31.715685 6566 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac648cca89d043d1a4bf656d800467b3" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37517 } }
I20260812 03:56:31.715266 6560 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:31.716873 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.711289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54930 (local address 127.17.54.254:37517)
0812 03:56:31.711510 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.711518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.711544 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:56:31.712020 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.712027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.712085 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.712200 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:31.712492 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.713408 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.714532 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.715133 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.715289 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.716081 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:31.716084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:31.716085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:31.716324 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:31.716371 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:31.716402 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:31.716467 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.716563 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.716661 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260812 03:56:31.717700 6568 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac648cca89d043d1a4bf656d800467b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac648cca89d043d1a4bf656d800467b3" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37517 } } }
I20260812 03:56:31.717844 6567 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac648cca89d043d1a4bf656d800467b3. Latest consensus state: current_term: 1 leader_uuid: "ac648cca89d043d1a4bf656d800467b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac648cca89d043d1a4bf656d800467b3" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37517 } } }
I20260812 03:56:31.717898 6567 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:31.717895 6568 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:31.718204 6581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:31.718647 6581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:31.720384 6581 catalog_manager.cc:1347] Generated new cluster ID: 2d2cc25b0c964210b19972d826bef0c9
I20260812 03:56:31.720492 6581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:31.726344 6581 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:31.726905 6581 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:31.734756 6581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3: Generated new TSK 0
I20260812 03:56:31.734989 6581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:31.741900 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37517
--builtin_ntp_servers=127.17.54.212:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:56:31.848464 6585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:31.848675 6585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:31.848697 6585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:31.849896 6585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:31.849941 6585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:31.849967 6585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:56:31.851467 6585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:31.851713 6585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:31.852034 6585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:31.858368 6593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:31.858353 6590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:31.858593 6585 server_base.cc:1034] running on GCE node
W20260812 03:56:31.858347 6591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:31.858978 6585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:31.859297 6585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:31.860471 6585 hybrid_clock.cc:648] HybridClock initialized: now 1786506991860437 us; error 52 us; skew 500 ppm
I20260812 03:56:31.862003 6585 webserver.cc:460] Webserver started at http://127.17.54.193:36009/ using document root <none> and password file <none>
I20260812 03:56:31.862282 6585 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:31.862401 6585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:31.862561 6585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:31.863726 6585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "ab1d90d1bb1b46099f098829341f5507"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "82cb812d5037115661f0cda56f283bf1"
server_key_iv: "01c03834403bf704402a68e14eb31c87"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.864107 6585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "ab1d90d1bb1b46099f098829341f5507"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "82cb812d5037115661f0cda56f283bf1"
server_key_iv: "01c03834403bf704402a68e14eb31c87"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.865608 6585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:31.866590 6603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.866786 6585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:31.866868 6585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "ab1d90d1bb1b46099f098829341f5507"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "82cb812d5037115661f0cda56f283bf1"
server_key_iv: "01c03834403bf704402a68e14eb31c87"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.866940 6585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:31.880616 6585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:31.880985 6585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:31.881124 6585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:31.881374 6585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:31.881726 6585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:31.881765 6585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.881806 6585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:31.881830 6585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.889209 6585 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:41997
I20260812 03:56:31.889269 6728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:41997 every 8 connection(s)
I20260812 03:56:31.889626 6585 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:56:31.890956 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 6585
I20260812 03:56:31.891075 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:56:31.891340 17627 external_mini_cluster.cc:1428] Setting key a8e1ab077a1d3b7c4bdae78f450211db
I20260812 03:56:31.892462 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37517
--builtin_ntp_servers=127.17.54.212:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:56:31.896039 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.890466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:50981 (local address 127.17.54.254:37517)
0812 03:56:31.890558 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:31.890560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:31.890889 (+ 329us) server_negotiation.cc:407] Connection header received
0812 03:56:31.891621 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:31.891622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:31.891639 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:31.891697 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:31.892509 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.893106 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.894576 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.894802 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.894885 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.895770 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:31.895771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:31.895772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:31.895777 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:31.895793 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:31.895796 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:31.895845 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:31.895936 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 03:56:31.895960 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:56:31.896324 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:31.890479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37517 (local address 127.17.54.193:50981)
0812 03:56:31.890673 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:31.890739 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:31.891507 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:31.891754 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:31.891757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:31.891766 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:31.892394 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.892401 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.893217 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.893219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.894484 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:31.894487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:31.894896 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:31.894898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:31.895028 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:31.895609 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:31.895609 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:31.895648 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:31.895679 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:31.895860 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:31.895865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:31.895868 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:31.895939 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:56:31.896055 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260812 03:56:31.896881 6729 heartbeater.cc:347] Connected to a master server at 127.17.54.254:37517
I20260812 03:56:31.896987 6729 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:31.897197 6729 heartbeater.cc:511] Master 127.17.54.254:37517 requested a full tablet report, sending...
I20260812 03:56:31.897816 6522 ts_manager.cc:194] Registered new tserver with Master: ab1d90d1bb1b46099f098829341f5507 (127.17.54.193:41997)
I20260812 03:56:31.898844 6522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.193:50981
W20260812 03:56:31.979643 6732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:31.979848 6732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:31.979871 6732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:31.980991 6732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:31.981025 6732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:31.981050 6732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:56:31.982388 6732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:31.982601 6732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:31.982867 6732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:31.988370 6739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:31.988370 6738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:31.988390 6742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:31.988611 6732 server_base.cc:1034] running on GCE node
I20260812 03:56:31.988837 6732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:31.989152 6732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:31.990376 6732 hybrid_clock.cc:648] HybridClock initialized: now 1786506991990361 us; error 31 us; skew 500 ppm
I20260812 03:56:31.991907 6732 webserver.cc:460] Webserver started at http://127.17.54.194:43033/ using document root <none> and password file <none>
I20260812 03:56:31.992170 6732 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:31.992214 6732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:31.992309 6732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:31.993381 6732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "c7f1a4e38feb45898d31f7243b610c96"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "b4fbbc1f63cf2f50d5c3cf64d32e1f75"
server_key_iv: "ccb129cf0d8487dfff41f8391ea77a14"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.993692 6732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "c7f1a4e38feb45898d31f7243b610c96"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "b4fbbc1f63cf2f50d5c3cf64d32e1f75"
server_key_iv: "ccb129cf0d8487dfff41f8391ea77a14"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.995203 6732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:31.996083 6749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:31.996310 6732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:31.996443 6732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "c7f1a4e38feb45898d31f7243b610c96"
format_stamp: "Formatted at 2026-08-12 03:56:31 on dist-test-slave-tm1g"
server_key: "b4fbbc1f63cf2f50d5c3cf64d32e1f75"
server_key_iv: "ccb129cf0d8487dfff41f8391ea77a14"
server_key_version: "encryptionkey@0"
I20260812 03:56:31.996515 6732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:32.021421 6732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:32.021739 6732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:32.021849 6732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:32.022050 6732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:32.022462 6732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:32.022502 6732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:32.022552 6732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:32.022578 6732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:32.028524 6732 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:42217
I20260812 03:56:32.028636 6879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:42217 every 8 connection(s)
I20260812 03:56:32.028966 6732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:56:32.029256 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 6732
I20260812 03:56:32.029373 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:56:32.029623 17627 external_mini_cluster.cc:1428] Setting key 9ed1963549e5057affe9e54ef904355f
I20260812 03:56:32.030844 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37517
--builtin_ntp_servers=127.17.54.212:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:56:32.035661 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.029954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:60031 (local address 127.17.54.254:37517)
0812 03:56:32.030011 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.030013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.030222 (+ 209us) server_negotiation.cc:407] Connection header received
0812 03:56:32.031069 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.031071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.031090 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.031184 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:32.032212 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.032773 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.033971 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.034203 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.034295 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.035315 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:32.035317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:32.035317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:32.035325 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:32.035343 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:32.035347 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:32.035410 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.035530 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.035565 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:56:32.035903 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.029855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37517 (local address 127.17.54.194:60031)
0812 03:56:32.030071 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:32.030116 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:32.030886 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:32.031281 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:32.031286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:32.031298 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:32.032079 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.032087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.032912 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.032913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.033859 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.033862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.034335 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.034337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.034493 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.035144 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:32.035144 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:32.035178 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:32.035207 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:32.035413 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:32.035419 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:32.035422 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:32.035519 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:56:32.035639 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260812 03:56:32.036489 6880 heartbeater.cc:347] Connected to a master server at 127.17.54.254:37517
I20260812 03:56:32.036623 6880 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:32.036870 6880 heartbeater.cc:511] Master 127.17.54.254:37517 requested a full tablet report, sending...
I20260812 03:56:32.037437 6522 ts_manager.cc:194] Registered new tserver with Master: c7f1a4e38feb45898d31f7243b610c96 (127.17.54.194:42217)
I20260812 03:56:32.037845 6522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.194:60031
W20260812 03:56:32.118604 6883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:32.118875 6883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:32.118917 6883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:32.120160 6883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:32.120232 6883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:32.120277 6883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:56:32.121758 6883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:32.122071 6883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:32.122450 6883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:32.128041 6890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:32.128033 6892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:32.128003 6888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:32.128476 6883 server_base.cc:1034] running on GCE node
I20260812 03:56:32.128758 6883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:32.129092 6883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:32.130282 6883 hybrid_clock.cc:648] HybridClock initialized: now 1786506992130285 us; error 51 us; skew 500 ppm
I20260812 03:56:32.131922 6883 webserver.cc:460] Webserver started at http://127.17.54.195:32861/ using document root <none> and password file <none>
I20260812 03:56:32.132225 6883 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:32.132299 6883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:32.132493 6883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:32.133636 6883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "aa7bc73db8754aedbaed546f133fb9d4"
format_stamp: "Formatted at 2026-08-12 03:56:32 on dist-test-slave-tm1g"
server_key: "3f222809f8300a12057a7932fb23eb1b"
server_key_iv: "a412af4334e6af8f788d06b3d5f4df55"
server_key_version: "encryptionkey@0"
I20260812 03:56:32.134022 6883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "aa7bc73db8754aedbaed546f133fb9d4"
format_stamp: "Formatted at 2026-08-12 03:56:32 on dist-test-slave-tm1g"
server_key: "3f222809f8300a12057a7932fb23eb1b"
server_key_iv: "a412af4334e6af8f788d06b3d5f4df55"
server_key_version: "encryptionkey@0"
I20260812 03:56:32.135551 6883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:32.136459 6898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:32.136703 6883 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:32.136812 6883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "aa7bc73db8754aedbaed546f133fb9d4"
format_stamp: "Formatted at 2026-08-12 03:56:32 on dist-test-slave-tm1g"
server_key: "3f222809f8300a12057a7932fb23eb1b"
server_key_iv: "a412af4334e6af8f788d06b3d5f4df55"
server_key_version: "encryptionkey@0"
I20260812 03:56:32.136929 6883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:32.144644 6883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:32.144979 6883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:32.145140 6883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:32.145390 6883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:32.145738 6883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:32.145797 6883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:32.145843 6883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:32.145867 6883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:32.152103 6883 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:38897
I20260812 03:56:32.152149 7025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:38897 every 8 connection(s)
I20260812 03:56:32.152515 6883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:56:32.157389 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 6883
I20260812 03:56:32.157511 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:56:32.157783 17627 external_mini_cluster.cc:1428] Setting key 15080223d21a20382f505318d109c131
I20260812 03:56:32.158221 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.153334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:53797 (local address 127.17.54.254:37517)
0812 03:56:32.153418 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.153420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.153650 (+ 230us) server_negotiation.cc:407] Connection header received
0812 03:56:32.154193 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.154199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.154211 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.154287 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:32.155150 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.155685 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.156660 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.156846 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.156912 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.157927 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:32.157929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:32.157929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:32.157936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:32.157954 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:32.157958 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:32.158033 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.158110 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.158140 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:56:32.158502 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.153304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37517 (local address 127.17.54.195:53797)
0812 03:56:32.153498 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:32.153544 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:32.154087 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:32.154428 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:32.154438 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:32.154447 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:32.155051 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.155056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.155773 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.155774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.156580 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.156582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.156953 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.156954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.157085 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.157756 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:32.157757 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:32.157789 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:32.157843 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:32.158037 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:32.158043 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:32.158046 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:32.158115 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:56:32.158221 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":101,"threads_started":1}
I20260812 03:56:32.158991 7026 heartbeater.cc:347] Connected to a master server at 127.17.54.254:37517
I20260812 03:56:32.159086 7026 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:32.159293 7026 heartbeater.cc:511] Master 127.17.54.254:37517 requested a full tablet report, sending...
I20260812 03:56:32.159690 6522 ts_manager.cc:194] Registered new tserver with Master: aa7bc73db8754aedbaed546f133fb9d4 (127.17.54.195:38897)
I20260812 03:56:32.160069 6522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.195:53797
I20260812 03:56:32.170066 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:56:32.175208 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.171896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54940 (local address 127.17.54.254:37517)
0812 03:56:32.171966 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.171968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.172021 (+ 53us) server_negotiation.cc:407] Connection header received
0812 03:56:32.172113 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.172114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.172125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.172183 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:32.172363 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.172855 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.173804 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.174005 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.174064 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.174906 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:32.174907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:32.174907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:32.174912 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:32.174926 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:32.174930 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:32.175025 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.175129 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.175142 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:56:32.177342 6522 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:56:32.179108 6522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:56:32.185616 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.181422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:41997 (local address 127.0.0.1:37576)
0812 03:56:32.181567 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:32.181610 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:32.181675 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:32.182328 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:32.182329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:32.182331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:32.182413 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.182415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.183351 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.183352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.184457 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.184460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.185277 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.185278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.185375 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.185389 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:56:32.185474 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:56:32.185534 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260812 03:56:32.185627 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.181639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.17.54.193:41997)
0812 03:56:32.182101 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.182106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.182139 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:56:32.182183 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.182185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.182202 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.182500 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:32.182560 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.183252 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.184556 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.185183 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.185278 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.185395 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.185495 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.185534 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":103,"threads_started":1}
I20260812 03:56:32.186120 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.181656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:42217 (local address 127.0.0.1:50346)
0812 03:56:32.181892 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:32.181900 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:32.181937 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:32.182294 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:32.182297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:32.182322 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:32.182408 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.182411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.183909 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.183910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.184848 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.184850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.185892 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.185894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.185954 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.185958 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:56:32.185986 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 03:56:32.186026 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":73,"threads_started":1}
I20260812 03:56:32.186491 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.181917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50346 (local address 127.17.54.194:42217)
0812 03:56:32.182087 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.182093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.182104 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:56:32.182147 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.182149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.182165 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.182492 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:32.182556 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.183811 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.184934 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.185647 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.186240 (+ 593us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.186362 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.186387 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.186431 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260812 03:56:32.186532 6657 tablet_service.cc:1467] Processing CreateTablet for tablet 14a8922fdc8a4ec7b341f4a2de140ec1 (DEFAULT_TABLE table=test-table [id=61788c78d184415483177f190597eb20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:32.186975 6657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a8922fdc8a4ec7b341f4a2de140ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:32.187076 6804 tablet_service.cc:1467] Processing CreateTablet for tablet 14a8922fdc8a4ec7b341f4a2de140ec1 (DEFAULT_TABLE table=test-table [id=61788c78d184415483177f190597eb20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:32.187414 6804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a8922fdc8a4ec7b341f4a2de140ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:32.187445 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.181426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:38897 (local address 127.0.0.1:58834)
0812 03:56:32.181601 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:32.181612 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:32.181675 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:32.182410 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:32.182411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:32.182413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:32.182451 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.182453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.184953 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.184954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.186053 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.186055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.187117 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.187119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.187184 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.187189 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:56:32.187259 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:56:32.187357 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260812 03:56:32.187567 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.181871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.17.54.195:38897)
0812 03:56:32.182093 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.182097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.182108 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:56:32.182148 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.182149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.182165 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.182251 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:32.182533 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.183810 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.186120 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.186951 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.187031 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.187117 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.187319 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.187372 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
I20260812 03:56:32.188279 6949 tablet_service.cc:1467] Processing CreateTablet for tablet 14a8922fdc8a4ec7b341f4a2de140ec1 (DEFAULT_TABLE table=test-table [id=61788c78d184415483177f190597eb20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:32.188599 6949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a8922fdc8a4ec7b341f4a2de140ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:32.189977 7042 tablet_bootstrap.cc:492] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Bootstrap starting.
I20260812 03:56:32.190483 7043 tablet_bootstrap.cc:492] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507: Bootstrap starting.
I20260812 03:56:32.190737 7042 tablet_bootstrap.cc:654] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:32.191083 7042 log.cc:824] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:32.191154 7043 tablet_bootstrap.cc:654] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:32.191447 7043 log.cc:824] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:32.191967 7042 tablet_bootstrap.cc:492] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: No bootstrap required, opened a new log
I20260812 03:56:32.191949 7046 tablet_bootstrap.cc:492] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Bootstrap starting.
I20260812 03:56:32.192032 7042 ts_tablet_manager.cc:1397] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:56:32.192582 7046 tablet_bootstrap.cc:654] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:32.192917 7046 log.cc:824] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:32.193675 7046 tablet_bootstrap.cc:492] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: No bootstrap required, opened a new log
I20260812 03:56:32.193742 7046 ts_tablet_manager.cc:1397] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:56:32.194912 7043 tablet_bootstrap.cc:492] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507: No bootstrap required, opened a new log
I20260812 03:56:32.194864 7042 raft_consensus.cc:348] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.195012 7043 ts_tablet_manager.cc:1397] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 03:56:32.195034 7042 raft_consensus.cc:374] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:32.195076 7042 raft_consensus.cc:729] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f1a4e38feb45898d31f7243b610c96, State: Initialized, Role: FOLLOWER
I20260812 03:56:32.195217 7042 consensus_queue.cc:260] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.195500 7042 ts_tablet_manager.cc:1428] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260812 03:56:32.195629 6880 heartbeater.cc:503] Master 127.17.54.254:37517 was elected leader, sending a full tablet report...
I20260812 03:56:32.196360 7046 raft_consensus.cc:348] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.196528 7046 raft_consensus.cc:374] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:32.196568 7046 raft_consensus.cc:729] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa7bc73db8754aedbaed546f133fb9d4, State: Initialized, Role: FOLLOWER
I20260812 03:56:32.196692 7046 consensus_queue.cc:260] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.196991 7046 ts_tablet_manager.cc:1428] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 03:56:32.197029 7026 heartbeater.cc:503] Master 127.17.54.254:37517 was elected leader, sending a full tablet report...
I20260812 03:56:32.197484 7043 raft_consensus.cc:348] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.197626 7043 raft_consensus.cc:374] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:32.197670 7043 raft_consensus.cc:729] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab1d90d1bb1b46099f098829341f5507, State: Initialized, Role: FOLLOWER
I20260812 03:56:32.197786 7043 consensus_queue.cc:260] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.198060 7043 ts_tablet_manager.cc:1428] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:56:32.198144 6729 heartbeater.cc:503] Master 127.17.54.254:37517 was elected leader, sending a full tablet report...
W20260812 03:56:32.279747 6881 tablet.cc:2369] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:56:32.369717 7050 raft_consensus.cc:482] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:32.369850 7050 raft_consensus.cc:504] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.370203 7050 leader_election.cc:288] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab1d90d1bb1b46099f098829341f5507 (127.17.54.193:41997), c7f1a4e38feb45898d31f7243b610c96 (127.17.54.194:42217)
I20260812 03:56:32.374487 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.370430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:41997 (local address 127.17.54.195:45821)
0812 03:56:32.370474 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:32.370489 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:32.370535 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:32.370857 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:32.370858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:32.370860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:32.370954 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.370956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.371988 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.371989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.373342 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.373346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.374140 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.374141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.374239 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.374244 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:56:32.374331 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:56:32.374387 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":44}
I20260812 03:56:32.374548 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.370555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:45821 (local address 127.17.54.193:41997)
0812 03:56:32.370607 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.370610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.370626 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:56:32.370652 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.370652 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.370680 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.370747 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:32.371026 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.371868 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.373470 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.374027 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.374115 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.374124 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.374423 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.374474 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:56:32.374548 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.370751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:53461 (local address 127.17.54.194:42217)
0812 03:56:32.370826 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.370828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.370847 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:56:32.370926 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.370927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.370939 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.370992 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:32.371150 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.371871 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.373470 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.374046 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.374115 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.374124 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.374423 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.374476 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:56:32.374493 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.370554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:42217 (local address 127.17.54.195:53461)
0812 03:56:32.370789 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:32.370800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:32.370851 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:32.370999 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:32.371000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:32.371002 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:32.371071 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.371072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.372008 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.372009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.373344 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:32.373346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.374140 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.374141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:32.374247 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.374249 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:56:32.374378 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 03:56:32.374410 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":57,"thread_start_us":112,"threads_started":1}
I20260812 03:56:32.374991 6678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a8922fdc8a4ec7b341f4a2de140ec1" candidate_uuid: "aa7bc73db8754aedbaed546f133fb9d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab1d90d1bb1b46099f098829341f5507" is_pre_election: true
I20260812 03:56:32.375139 6678 raft_consensus.cc:2454] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa7bc73db8754aedbaed546f133fb9d4 in term 0.
I20260812 03:56:32.375391 6899 leader_election.cc:302] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa7bc73db8754aedbaed546f133fb9d4, ab1d90d1bb1b46099f098829341f5507; no voters:
I20260812 03:56:32.375558 7050 raft_consensus.cc:2781] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:56:32.375626 7050 raft_consensus.cc:482] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:56:32.375655 7050 raft_consensus.cc:3037] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:32.375593 6828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a8922fdc8a4ec7b341f4a2de140ec1" candidate_uuid: "aa7bc73db8754aedbaed546f133fb9d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1a4e38feb45898d31f7243b610c96" is_pre_election: true
I20260812 03:56:32.375787 6828 raft_consensus.cc:2454] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa7bc73db8754aedbaed546f133fb9d4 in term 0.
I20260812 03:56:32.376552 7050 raft_consensus.cc:504] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.376699 7050 leader_election.cc:288] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [CANDIDATE]: Term 1 election: Requested vote from peers ab1d90d1bb1b46099f098829341f5507 (127.17.54.193:41997), c7f1a4e38feb45898d31f7243b610c96 (127.17.54.194:42217)
I20260812 03:56:32.376866 6678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a8922fdc8a4ec7b341f4a2de140ec1" candidate_uuid: "aa7bc73db8754aedbaed546f133fb9d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab1d90d1bb1b46099f098829341f5507"
I20260812 03:56:32.376875 6828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a8922fdc8a4ec7b341f4a2de140ec1" candidate_uuid: "aa7bc73db8754aedbaed546f133fb9d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f1a4e38feb45898d31f7243b610c96"
I20260812 03:56:32.376951 6678 raft_consensus.cc:3037] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:32.376984 6828 raft_consensus.cc:3037] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:32.377714 6678 raft_consensus.cc:2454] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa7bc73db8754aedbaed546f133fb9d4 in term 1.
I20260812 03:56:32.377882 6828 raft_consensus.cc:2454] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa7bc73db8754aedbaed546f133fb9d4 in term 1.
I20260812 03:56:32.377887 6899 leader_election.cc:302] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa7bc73db8754aedbaed546f133fb9d4, ab1d90d1bb1b46099f098829341f5507; no voters:
I20260812 03:56:32.378101 7050 raft_consensus.cc:2781] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:32.378253 7050 raft_consensus.cc:686] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 1 LEADER]: Becoming Leader. State: Replica: aa7bc73db8754aedbaed546f133fb9d4, State: Running, Role: LEADER
I20260812 03:56:32.378381 7050 consensus_queue.cc:237] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } }
I20260812 03:56:32.379246 6521 catalog_manager.cc:5495] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa7bc73db8754aedbaed546f133fb9d4 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "aa7bc73db8754aedbaed546f133fb9d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa7bc73db8754aedbaed546f133fb9d4" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 38897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 } health_report { overall_health: UNKNOWN } } }
W20260812 03:56:32.390506 6730 tablet.cc:2369] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:56:32.403220 7027 tablet.cc:2369] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:56:32.419693 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.417161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.17.54.195:38897)
0812 03:56:32.417224 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:32.417228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:32.417241 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:56:32.417296 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:32.417296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:32.417316 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:32.417382 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:56:32.417519 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.418009 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.418943 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:32.419146 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:32.419201 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:32.419387 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:56:32.419482 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:56:32.419555 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:32.419605 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 03:56:32.419614 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:56:32.421762 6828 raft_consensus.cc:1261] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 1 FOLLOWER]: Refusing update from remote peer aa7bc73db8754aedbaed546f133fb9d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:32.421813 6678 raft_consensus.cc:1261] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [term 1 FOLLOWER]: Refusing update from remote peer aa7bc73db8754aedbaed546f133fb9d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:32.422168 7050 consensus_queue.cc:1035] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f1a4e38feb45898d31f7243b610c96" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 42217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:32.422324 7053 consensus_queue.cc:1035] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab1d90d1bb1b46099f098829341f5507" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 41997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:32.425009 7057 mvcc.cc:205] Tried to move back new op lower bound from 7317532640956243968 to 7317532640782610432. Current Snapshot: MvccSnapshot[applied={T|T < 7317532640956243968}]
I20260812 03:56:32.427609 6521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:32.427714 6521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:32.428615 6521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac648cca89d043d1a4bf656d800467b3: Sending DeleteTablet for 3 replicas of tablet 14a8922fdc8a4ec7b341f4a2de140ec1
I20260812 03:56:32.429028 6949 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a8922fdc8a4ec7b341f4a2de140ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:32 UTC) from {username='slave'} at 127.0.0.1:58834
I20260812 03:56:32.429068 6804 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a8922fdc8a4ec7b341f4a2de140ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:32 UTC) from {username='slave'} at 127.0.0.1:50346
I20260812 03:56:32.429271 6657 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a8922fdc8a4ec7b341f4a2de140ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:32 UTC) from {username='slave'} at 127.0.0.1:37576
I20260812 03:56:32.429507 7068 tablet_replica.cc:330] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: stopping tablet replica
I20260812 03:56:32.429602 7068 raft_consensus.cc:2229] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:56:32.429584 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 6585
I20260812 03:56:32.429697 7068 raft_consensus.cc:2258] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:32.429802 7070 tablet_replica.cc:330] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507: stopping tablet replica
I20260812 03:56:32.429965 7070 raft_consensus.cc:2229] T 14a8922fdc8a4ec7b341f4a2de140ec1 P ab1d90d1bb1b46099f098829341f5507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:32.430285 7069 tablet_replica.cc:330] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: stopping tablet replica
I20260812 03:56:32.430408 7069 raft_consensus.cc:2229] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:32.430513 7069 raft_consensus.cc:2258] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:32.431706 7068 ts_tablet_manager.cc:1905] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:32.433701 7068 ts_tablet_manager.cc:1918] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:32.433801 7068 log.cc:1196] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/14a8922fdc8a4ec7b341f4a2de140ec1
I20260812 03:56:32.434077 7068 ts_tablet_manager.cc:1939] T 14a8922fdc8a4ec7b341f4a2de140ec1 P aa7bc73db8754aedbaed546f133fb9d4: Deleting consensus metadata
I20260812 03:56:32.434734 6507 catalog_manager.cc:4841] TS aa7bc73db8754aedbaed546f133fb9d4 (127.17.54.195:38897): tablet 14a8922fdc8a4ec7b341f4a2de140ec1 (table test-table [id=61788c78d184415483177f190597eb20]) successfully deleted
I20260812 03:56:32.441759 7069 ts_tablet_manager.cc:1905] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:32.444311 7069 ts_tablet_manager.cc:1918] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:32.444424 7069 log.cc:1196] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/14a8922fdc8a4ec7b341f4a2de140ec1
I20260812 03:56:32.444701 7069 ts_tablet_manager.cc:1939] T 14a8922fdc8a4ec7b341f4a2de140ec1 P c7f1a4e38feb45898d31f7243b610c96: Deleting consensus metadata
I20260812 03:56:32.445283 6503 catalog_manager.cc:4841] TS c7f1a4e38feb45898d31f7243b610c96 (127.17.54.194:42217): tablet 14a8922fdc8a4ec7b341f4a2de140ec1 (table test-table [id=61788c78d184415483177f190597eb20]) successfully deleted
W20260812 03:56:32.446669 6503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:41997 (error 108)
I20260812 03:56:32.446895 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 6732
I20260812 03:56:32.449767 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:32.448634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:41997 (local address 127.0.0.1:37584)
0812 03:56:32.449659 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:32.449725 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:41997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":992}
W20260812 03:56:32.450476 6503 catalog_manager.cc:4584] TS ab1d90d1bb1b46099f098829341f5507 (127.17.54.193:41997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14a8922fdc8a4ec7b341f4a2de140ec1: Network error: Client connection negotiation failed: client connection to 127.17.54.193:41997: connect: Connection refused (error 111)
I20260812 03:56:32.455858 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 6883
I20260812 03:56:32.463811 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 6485
2026-08-12T03:56:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (926 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:32 dist-test-slave-tm1g krb5kdc[7074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:32 dist-test-slave-tm1g krb5kdc[7074](info): set up 2 sockets
Aug 12 03:56:32 dist-test-slave-tm1g krb5kdc[7074](info): commencing operation
krb5kdc: starting...
W20260812 03:56:33.594942 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:33 dist-test-slave-tm1g krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:56:33.730773 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:40259
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:40259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:33.816820 7087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:33.817072 7087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:33.817111 7087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:33.818275 7087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:33.818372 7087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:33.818409 7087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:33.818441 7087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:33.818463 7087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:33.819976 7087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:40259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:40259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:33.820266 7087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:33.820508 7087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:33.826004 7093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:33.826131 7087 server_base.cc:1034] running on GCE node
W20260812 03:56:33.826011 7096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:33.826011 7092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:33.826574 7087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:33.826930 7087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:33.828157 7087 hybrid_clock.cc:648] HybridClock initialized: now 1786506993828130 us; error 45 us; skew 500 ppm
Aug 12 03:56:33 dist-test-slave-tm1g krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506993, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:33.830925 7087 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:33.831435 7087 webserver.cc:460] Webserver started at http://127.17.54.254:35937/ using document root <none> and password file <none>
I20260812 03:56:33.831667 7087 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:33.831736 7087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:33.831897 7087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:33.833022 7087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "5212f73a03f2410287aaf7a813a82fa5"
format_stamp: "Formatted at 2026-08-12 03:56:33 on dist-test-slave-tm1g"
server_key: "e6fdf7dc69e59a6f8eed627012a44837"
server_key_iv: "c1ad50e41c80b57236f937acdf8d1148"
server_key_version: "encryptionkey@0"
I20260812 03:56:33.833391 7087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "5212f73a03f2410287aaf7a813a82fa5"
format_stamp: "Formatted at 2026-08-12 03:56:33 on dist-test-slave-tm1g"
server_key: "e6fdf7dc69e59a6f8eed627012a44837"
server_key_iv: "c1ad50e41c80b57236f937acdf8d1148"
server_key_version: "encryptionkey@0"
I20260812 03:56:33.834867 7087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:33.835729 7104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:33.835978 7087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:33.836098 7087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "5212f73a03f2410287aaf7a813a82fa5"
format_stamp: "Formatted at 2026-08-12 03:56:33 on dist-test-slave-tm1g"
server_key: "e6fdf7dc69e59a6f8eed627012a44837"
server_key_iv: "c1ad50e41c80b57236f937acdf8d1148"
server_key_version: "encryptionkey@0"
I20260812 03:56:33.836175 7087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:33.843545 7087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:33.845849 7087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:33.846010 7087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:33.850382 7087 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:40259
I20260812 03:56:33.850457 7169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:40259 every 8 connection(s)
I20260812 03:56:33.850821 7087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:33.851810 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:33.854607 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5: Bootstrap starting.
I20260812 03:56:33.855293 7170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:33.855643 7170 log.cc:824] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:33.856539 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5: No bootstrap required, opened a new log
I20260812 03:56:33.856837 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 7087
I20260812 03:56:33.857007 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:33.857220 17627 external_mini_cluster.cc:1428] Setting key ccd7ddf643cfb045a4c7485a388e621d
I20260812 03:56:33.859112 7170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5212f73a03f2410287aaf7a813a82fa5" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 40259 } }
I20260812 03:56:33.859301 7170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:33.859365 7170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5212f73a03f2410287aaf7a813a82fa5, State: Initialized, Role: FOLLOWER
I20260812 03:56:33.859539 7170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5212f73a03f2410287aaf7a813a82fa5" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 40259 } }
I20260812 03:56:33.859642 7170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:33.859695 7170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:33.859779 7170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:33.860867 7170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5212f73a03f2410287aaf7a813a82fa5" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 40259 } }
I20260812 03:56:33.861004 7170 leader_election.cc:302] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5212f73a03f2410287aaf7a813a82fa5; no voters:
I20260812 03:56:33.861229 7170 leader_election.cc:288] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:33.861256 7175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:33.861450 7175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5212f73a03f2410287aaf7a813a82fa5, State: Running, Role: LEADER
I20260812 03:56:33.861541 7175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5212f73a03f2410287aaf7a813a82fa5" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 40259 } }
I20260812 03:56:33.861560 7170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:33.862089 7177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5212f73a03f2410287aaf7a813a82fa5. Latest consensus state: current_term: 1 leader_uuid: "5212f73a03f2410287aaf7a813a82fa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5212f73a03f2410287aaf7a813a82fa5" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 40259 } } }
I20260812 03:56:33.862092 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5212f73a03f2410287aaf7a813a82fa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5212f73a03f2410287aaf7a813a82fa5" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 40259 } } }
I20260812 03:56:33.862231 7177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:33.862241 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:33.862483 7181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:33.863058 7181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:33.865000 7181 catalog_manager.cc:1347] Generated new cluster ID: bba5ba8eead94558a4b3dcae962a4bec
I20260812 03:56:33.865064 7181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:56:33 dist-test-slave-tm1g krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:33.870048 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:33.857799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59634 (local address 127.17.54.254:40259)
0812 03:56:33.858005 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:33.858017 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:33.858050 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:56:33.859139 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:33.859147 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:33.859203 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:33.859385 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:33.860208 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:33.861213 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:33.862350 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:33.863058 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:33.863151 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:33.866550 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:33.866556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:33.866556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:33.866574 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:33.868289 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:33.868679 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:33.868682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:33.868682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:33.868804 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:33.869034 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:33.869035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:33.869035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:33.869284 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:33.869465 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:33.869545 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:56:33.869696 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
I20260812 03:56:33.878736 7181 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:33.879391 7181 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:33.888127 7181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5: Generated new TSK 0
I20260812 03:56:33.888315 7181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.54.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
Entry for principal kudu/127.17.54.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab.
I20260812 03:56:33.925735 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:40259
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:34.028908 7197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:34.029156 7197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:34.029186 7197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:34.030640 7197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:34.030699 7197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:34.030731 7197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:56:34.032534 7197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.193.keytab
--principal=kudu/127.17.54.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:40259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:34.032836 7197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:34.033151 7197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:34.038740 7205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:34.038745 7204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:34.038753 7208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:34.039273 7197 server_base.cc:1034] running on GCE node
I20260812 03:56:34.039497 7197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:34.039810 7197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:34.040983 7197 hybrid_clock.cc:648] HybridClock initialized: now 1786506994040976 us; error 47 us; skew 500 ppm
Aug 12 03:56:34 dist-test-slave-tm1g krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506994, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:34.043740 7197 init.cc:377] Logged in from keytab as kudu/127.17.54.193@KRBTEST.COM (short username kudu)
I20260812 03:56:34.044377 7197 webserver.cc:460] Webserver started at http://127.17.54.193:34051/ using document root <none> and password file <none>
I20260812 03:56:34.044596 7197 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:34.044641 7197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:34.044734 7197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:34.045814 7197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "8227dfe9d9684ac3b3618a5d417d172c"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "26d224b50534f03cc742236f0bec89cc"
server_key_iv: "67f28939012f3974ec065816d9981c44"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.046125 7197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "8227dfe9d9684ac3b3618a5d417d172c"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "26d224b50534f03cc742236f0bec89cc"
server_key_iv: "67f28939012f3974ec065816d9981c44"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.047645 7197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:34.048514 7215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.048754 7197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:34.048848 7197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "8227dfe9d9684ac3b3618a5d417d172c"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "26d224b50534f03cc742236f0bec89cc"
server_key_iv: "67f28939012f3974ec065816d9981c44"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.048910 7197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:34.096935 7197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:34.099326 7197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:34.099444 7197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:34.099660 7197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:34.099984 7197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:34.100021 7197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.100044 7197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:34.100056 7197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.105948 7197 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:36427
I20260812 03:56:34.106032 7346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:36427 every 8 connection(s)
I20260812 03:56:34.106411 7197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:56:34 dist-test-slave-tm1g krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506994, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.193@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:34.112818 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 7197
I20260812 03:56:34.112967 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:56:34.113191 17627 external_mini_cluster.cc:1428] Setting key 0cf80e9f2f1eda16ed68094521c6a3e6
I20260812 03:56:34.116153 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.107171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:50243 (local address 127.17.54.254:40259)
0812 03:56:34.107221 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.107223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.107448 (+ 225us) server_negotiation.cc:407] Connection header received
0812 03:56:34.107944 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.107946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.107957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.108017 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:34.108914 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.109423 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.110469 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.110708 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.110770 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.112855 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:34.112857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:34.112857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:34.112867 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:34.114602 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:34.115019 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:34.115020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:34.115020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:34.115090 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:34.115392 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:34.115393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:34.115393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:34.115547 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:34.115618 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.115958 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.116074 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:56:34.116439 7349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.107119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:40259 (local address 127.17.54.193:50243)
0812 03:56:34.107328 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.107373 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:34.107850 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:34.108075 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:34.108078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:34.108634 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:34.108828 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.108833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.109509 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.109509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.110382 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.110385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.110787 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.110787 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.110887 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.111411 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:34.111420 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:34.112754 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:34.114697 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:34.114704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:34.114705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:34.114940 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:34.115189 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:34.115190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:34.115190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:34.115311 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:34.115620 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:34.115627 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:34.115759 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 03:56:34.115928 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 03:56:34.116147 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260812 03:56:34.117028 7347 heartbeater.cc:347] Connected to a master server at 127.17.54.254:40259
I20260812 03:56:34.117154 7347 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:34.117344 7347 heartbeater.cc:511] Master 127.17.54.254:40259 requested a full tablet report, sending...
I20260812 03:56:34.117904 7126 ts_manager.cc:194] Registered new tserver with Master: 8227dfe9d9684ac3b3618a5d417d172c (127.17.54.193:36427)
I20260812 03:56:34.118585 7126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.193@KRBTEST.COM'} at 127.17.54.193:50243
WARNING: no policy specified for kudu/127.17.54.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
Entry for principal kudu/127.17.54.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab.
I20260812 03:56:34.141448 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:40259
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:34.227721 7352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:34.227952 7352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:34.227990 7352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:34.229130 7352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:34.229195 7352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:34.229240 7352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:56:34.230783 7352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.194.keytab
--principal=kudu/127.17.54.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:40259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:34.231053 7352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:34.231279 7352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:34.236332 7358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:34.236409 7359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:34.236495 7352 server_base.cc:1034] running on GCE node
W20260812 03:56:34.236356 7361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:34.236739 7352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:34.236994 7352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:34.238175 7352 hybrid_clock.cc:648] HybridClock initialized: now 1786506994238157 us; error 39 us; skew 500 ppm
Aug 12 03:56:34 dist-test-slave-tm1g krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506994, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:34.240887 7352 init.cc:377] Logged in from keytab as kudu/127.17.54.194@KRBTEST.COM (short username kudu)
I20260812 03:56:34.241364 7352 webserver.cc:460] Webserver started at http://127.17.54.194:36119/ using document root <none> and password file <none>
I20260812 03:56:34.241611 7352 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:34.241679 7352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:34.241817 7352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:34.242987 7352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "22d2869b629741b6a5e760a3c34d39fc"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "2bc34074b300ab98fdf177cf4f62e604"
server_key_iv: "40be3e2abfcd68eeddd07d45174a0c18"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.243361 7352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "22d2869b629741b6a5e760a3c34d39fc"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "2bc34074b300ab98fdf177cf4f62e604"
server_key_iv: "40be3e2abfcd68eeddd07d45174a0c18"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.244817 7352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:56:34.245625 7368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.245805 7352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:34.245911 7352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "22d2869b629741b6a5e760a3c34d39fc"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "2bc34074b300ab98fdf177cf4f62e604"
server_key_iv: "40be3e2abfcd68eeddd07d45174a0c18"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.245991 7352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:34.259914 7352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:34.262534 7352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:34.262699 7352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:34.262946 7352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:34.263342 7352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:34.263415 7352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.263491 7352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:34.263520 7352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.269665 7352 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:45175
I20260812 03:56:34.269724 7502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:45175 every 8 connection(s)
I20260812 03:56:34.270077 7352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:56:34 dist-test-slave-tm1g krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506994, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.194@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:34.277237 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 7352
I20260812 03:56:34.277362 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:56:34.277576 17627 external_mini_cluster.cc:1428] Setting key 01e96a5e992a81b2d7db5de56548cc2e
I20260812 03:56:34.280071 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.270833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:57945 (local address 127.17.54.254:40259)
0812 03:56:34.270925 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.270927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.271159 (+ 232us) server_negotiation.cc:407] Connection header received
0812 03:56:34.271644 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.271644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.271655 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.271708 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:34.272623 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.273138 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.274133 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.274410 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.274489 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.276695 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:34.276697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:34.276697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:34.276704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:34.278489 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:34.279046 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:34.279047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:34.279047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:34.279106 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:34.279408 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:34.279409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:34.279409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:34.279528 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:34.279587 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.279926 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.280007 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:56:34.280373 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.270783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:40259 (local address 127.17.54.194:57945)
0812 03:56:34.270988 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.271047 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:34.271548 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:34.271763 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:34.271766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:34.272346 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:34.272541 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.272546 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.273224 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.273225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.274028 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.274031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.274508 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.274509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.274618 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.275121 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:34.275129 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:34.276582 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:34.278590 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:34.278598 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:34.278600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:34.278928 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:34.279181 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:34.279182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:34.279182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:34.279333 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:34.279599 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:34.279605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:34.279778 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 03:56:34.279895 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:56:34.280095 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260812 03:56:34.280925 7503 heartbeater.cc:347] Connected to a master server at 127.17.54.254:40259
I20260812 03:56:34.281045 7503 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:34.281226 7503 heartbeater.cc:511] Master 127.17.54.254:40259 requested a full tablet report, sending...
I20260812 03:56:34.281643 7126 ts_manager.cc:194] Registered new tserver with Master: 22d2869b629741b6a5e760a3c34d39fc (127.17.54.194:45175)
I20260812 03:56:34.282181 7126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.194@KRBTEST.COM'} at 127.17.54.194:57945
WARNING: no policy specified for kudu/127.17.54.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
Entry for principal kudu/127.17.54.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab.
I20260812 03:56:34.305877 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:40259
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:34.390390 7509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:34.390625 7509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:34.390667 7509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:34.391883 7509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:34.391954 7509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:34.392001 7509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:56:34.393513 7509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.195.keytab
--principal=kudu/127.17.54.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:40259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:34.393795 7509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:34.394052 7509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:34.399320 7516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:34.399377 7518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:34.399348 7515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:34.399582 7509 server_base.cc:1034] running on GCE node
I20260812 03:56:34.399770 7509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:34.400007 7509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:34.401163 7509 hybrid_clock.cc:648] HybridClock initialized: now 1786506994401140 us; error 34 us; skew 500 ppm
Aug 12 03:56:34 dist-test-slave-tm1g krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506994, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:34.403906 7509 init.cc:377] Logged in from keytab as kudu/127.17.54.195@KRBTEST.COM (short username kudu)
I20260812 03:56:34.404433 7509 webserver.cc:460] Webserver started at http://127.17.54.195:36437/ using document root <none> and password file <none>
I20260812 03:56:34.404667 7509 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:34.404735 7509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:34.404896 7509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:34.405979 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "98f353c6bf464faaa685c883ab892a40"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "eb631c4c469137aac944bfe5b8508fbd"
server_key_iv: "5331a820a1a904859eae2e106cbe4d34"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.406432 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "98f353c6bf464faaa685c883ab892a40"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "eb631c4c469137aac944bfe5b8508fbd"
server_key_iv: "5331a820a1a904859eae2e106cbe4d34"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.407861 7509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:34.408880 7525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.409209 7509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:34.409385 7509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "98f353c6bf464faaa685c883ab892a40"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "eb631c4c469137aac944bfe5b8508fbd"
server_key_iv: "5331a820a1a904859eae2e106cbe4d34"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.409492 7509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:34.439625 7509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:34.442397 7509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:34.442587 7509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:34.442875 7509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:34.443252 7509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:34.443313 7509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.443351 7509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:34.443401 7509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.449746 7509 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:34623
I20260812 03:56:34.449853 7664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:34623 every 8 connection(s)
I20260812 03:56:34.450191 7509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:56:34.451859 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 7509
I20260812 03:56:34.451967 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:56:34.452159 17627 external_mini_cluster.cc:1428] Setting key c14936666cbb1d80e36e95cf927aa597
Aug 12 03:56:34 dist-test-slave-tm1g krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506994, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.195@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:34.460497 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.451265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:34163 (local address 127.17.54.254:40259)
0812 03:56:34.451340 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.451342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.451480 (+ 138us) server_negotiation.cc:407] Connection header received
0812 03:56:34.452229 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.452231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.452246 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.452308 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:34.453513 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.454031 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.455131 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.455322 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.455382 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.457598 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:34.457599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:34.457599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:34.457606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:34.459213 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:34.459587 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:34.459588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:34.459588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:34.459644 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:34.459893 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:34.459894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:34.459895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:34.460016 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:34.460079 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.460364 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.460433 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:56:34.460613 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.451175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:40259 (local address 127.17.54.195:34163)
0812 03:56:34.451343 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.451387 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:34.452059 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:34.452379 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:34.452382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:34.453197 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:34.453402 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.453408 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.454138 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.454139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.455041 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.455044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.455421 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.455422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.455543 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.456040 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:56:34.456049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:34.457504 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:34.459288 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:34.459295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:34.459297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:34.459516 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:34.459712 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:56:34.459712 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:56:34.459713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:56:34.459809 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:56:34.460079 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:34.460084 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:34.460197 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 03:56:34.460281 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:56:34.460406 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260812 03:56:34.461107 7665 heartbeater.cc:347] Connected to a master server at 127.17.54.254:40259
I20260812 03:56:34.461217 7665 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:34.461400 7665 heartbeater.cc:511] Master 127.17.54.254:40259 requested a full tablet report, sending...
I20260812 03:56:34.461905 7126 ts_manager.cc:194] Registered new tserver with Master: 98f353c6bf464faaa685c883ab892a40 (127.17.54.195:34623)
I20260812 03:56:34.462268 7126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.54.195@KRBTEST.COM'} at 127.17.54.195:34163
I20260812 03:56:34.463932 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:56:34.471644 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.465424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.17.54.254:40259)
0812 03:56:34.465511 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.465513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.465624 (+ 111us) server_negotiation.cc:407] Connection header received
0812 03:56:34.465703 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.465703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.465714 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.465759 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:34.466339 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.466828 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.467592 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.467798 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.467881 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.469057 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:34.469058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:34.469059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:34.469065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:34.470428 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:34.470767 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:34.470768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:34.470768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:34.470833 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:34.471057 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:34.471058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:34.471058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:34.471224 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:34.471333 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.471461 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.471540 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:56:34.473344 7126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:56:34.475062 7126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:56:34.481757 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.477846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.17.54.194:45175)
0812 03:56:34.478072 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.478076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.478087 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:56:34.478127 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.478129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.478144 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.478225 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:34.478501 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.479296 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.480800 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.481320 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.481411 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.481477 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.481625 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.481670 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":109,"threads_started":1}
I20260812 03:56:34.481758 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.477718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:45175 (local address 127.0.0.1:48900)
0812 03:56:34.477929 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.477939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:34.477979 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:34.478352 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:34.478353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:34.478355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:34.478417 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.478419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.479403 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.479404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.480694 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.480697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.481414 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.481415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.481499 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.481516 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:56:34.481605 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:56:34.481662 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260812 03:56:34.481917 7677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.477512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36427 (local address 127.0.0.1:47624)
0812 03:56:34.477660 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.477703 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:34.477779 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:34.478082 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:34.478085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:34.478087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:34.478163 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.478165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.479663 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.479664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.480744 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.480746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.481664 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.481665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.481730 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.481734 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:56:34.481764 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 03:56:34.481816 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":4,"thread_start_us":65,"threads_started":1}
I20260812 03:56:34.482123 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.478097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43148 (local address 127.17.54.195:34623)
0812 03:56:34.478483 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.478488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.478497 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:56:34.478532 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.478534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.478548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.478627 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:34.478766 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.479761 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.480843 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.481557 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.481632 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.481689 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.481958 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.482007 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":95,"threads_started":1}
I20260812 03:56:34.481916 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.477891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:34623 (local address 127.0.0.1:43148)
0812 03:56:34.478286 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.478318 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:34.478366 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:34.478640 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:34.478641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:34.478643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:34.478677 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.478679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.479863 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.479864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.480768 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.480771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.481717 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.481717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.481773 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.481776 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:34.481820 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 03:56:34.481850 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":28,"thread_start_us":66,"threads_started":1}
I20260812 03:56:34.482403 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.477667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.17.54.193:36427)
0812 03:56:34.477892 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.477897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.477908 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:56:34.477952 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.477954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.477971 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.478247 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:34.478362 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.479558 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.480836 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.481559 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.482102 (+ 543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.482174 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.482200 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.482278 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":103,"threads_started":1}
I20260812 03:56:34.482944 7426 tablet_service.cc:1467] Processing CreateTablet for tablet 395f3886e5914d6c99164e68dc088bd7 (DEFAULT_TABLE table=test-table [id=485f840cc76a4ab2a27b289e64dad7b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:34.483099 7587 tablet_service.cc:1467] Processing CreateTablet for tablet 395f3886e5914d6c99164e68dc088bd7 (DEFAULT_TABLE table=test-table [id=485f840cc76a4ab2a27b289e64dad7b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:34.483100 7270 tablet_service.cc:1467] Processing CreateTablet for tablet 395f3886e5914d6c99164e68dc088bd7 (DEFAULT_TABLE table=test-table [id=485f840cc76a4ab2a27b289e64dad7b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:34.483238 7426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 395f3886e5914d6c99164e68dc088bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:34.483433 7270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 395f3886e5914d6c99164e68dc088bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:34.483433 7587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 395f3886e5914d6c99164e68dc088bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:34.485769 7683 tablet_bootstrap.cc:492] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Bootstrap starting.
I20260812 03:56:34.486521 7683 tablet_bootstrap.cc:654] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:34.487007 7685 tablet_bootstrap.cc:492] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: Bootstrap starting.
I20260812 03:56:34.487133 7683 log.cc:824] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:34.487635 7685 tablet_bootstrap.cc:654] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:34.487920 7683 tablet_bootstrap.cc:492] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: No bootstrap required, opened a new log
I20260812 03:56:34.487975 7683 ts_tablet_manager.cc:1397] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:56:34.488170 7685 log.cc:824] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:34.488523 7684 tablet_bootstrap.cc:492] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c: Bootstrap starting.
I20260812 03:56:34.489071 7684 tablet_bootstrap.cc:654] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:34.489563 7683 raft_consensus.cc:348] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.489686 7684 log.cc:824] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:34.489727 7683 raft_consensus.cc:374] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:34.489776 7683 raft_consensus.cc:729] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d2869b629741b6a5e760a3c34d39fc, State: Initialized, Role: FOLLOWER
I20260812 03:56:34.489904 7683 consensus_queue.cc:260] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.490224 7683 ts_tablet_manager.cc:1428] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:56:34.490257 7503 heartbeater.cc:503] Master 127.17.54.254:40259 was elected leader, sending a full tablet report...
I20260812 03:56:34.490307 7685 tablet_bootstrap.cc:492] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: No bootstrap required, opened a new log
I20260812 03:56:34.490381 7685 ts_tablet_manager.cc:1397] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:56:34.492187 7685 raft_consensus.cc:348] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.492316 7685 raft_consensus.cc:374] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:34.492338 7685 raft_consensus.cc:729] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f353c6bf464faaa685c883ab892a40, State: Initialized, Role: FOLLOWER
I20260812 03:56:34.492449 7685 consensus_queue.cc:260] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.492677 7685 ts_tablet_manager.cc:1428] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:56:34.492743 7665 heartbeater.cc:503] Master 127.17.54.254:40259 was elected leader, sending a full tablet report...
I20260812 03:56:34.493119 7684 tablet_bootstrap.cc:492] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c: No bootstrap required, opened a new log
I20260812 03:56:34.493194 7684 ts_tablet_manager.cc:1397] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:56:34.494861 7684 raft_consensus.cc:348] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.495002 7684 raft_consensus.cc:374] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:34.495025 7684 raft_consensus.cc:729] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8227dfe9d9684ac3b3618a5d417d172c, State: Initialized, Role: FOLLOWER
I20260812 03:56:34.495153 7684 consensus_queue.cc:260] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.495427 7684 ts_tablet_manager.cc:1428] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:56:34.495462 7347 heartbeater.cc:503] Master 127.17.54.254:40259 was elected leader, sending a full tablet report...
W20260812 03:56:34.520694 7504 tablet.cc:2369] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:56:34.607216 7348 tablet.cc:2369] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:56:34.651353 7690 raft_consensus.cc:482] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:34.651558 7690 raft_consensus.cc:504] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.652021 7690 leader_election.cc:288] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8227dfe9d9684ac3b3618a5d417d172c (127.17.54.193:36427), 22d2869b629741b6a5e760a3c34d39fc (127.17.54.194:45175)
I20260812 03:56:34.655139 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.652274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36427 (local address 127.17.54.195:58991)
0812 03:56:34.652320 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.652333 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:34.652384 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:34.652658 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:34.652659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:34.652662 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:34.652780 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.652782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.653471 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.653472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.654262 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.654265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.654860 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.654861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.654950 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.654953 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:34.655021 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:56:34.655065 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":33}
I20260812 03:56:34.655138 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.652350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:58991 (local address 127.17.54.193:36427)
0812 03:56:34.652470 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.652473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.652486 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:56:34.652526 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.652527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.652543 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.652622 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:34.652868 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.653377 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.654408 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.654773 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.654861 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.654871 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.655028 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.655065 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 03:56:34.655316 7692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.652408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:45175 (local address 127.17.54.195:43303)
0812 03:56:34.652609 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.652619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:34.652705 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:34.652856 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:34.652857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:34.652859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:34.652937 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.652941 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.653590 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.653591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.654454 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:34.654456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.655045 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.655045 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:34.655120 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.655124 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:56:34.655196 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:56:34.655237 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":15,"thread_start_us":142,"threads_started":1}
I20260812 03:56:34.655306 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.652558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:43303 (local address 127.17.54.194:45175)
0812 03:56:34.652607 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.652609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.652698 (+ 89us) server_negotiation.cc:407] Connection header received
0812 03:56:34.652784 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.652785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.652796 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.652838 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:34.653022 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.653496 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.654544 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.654939 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.655043 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.655053 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.655201 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.655243 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:56:34.655604 7448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395f3886e5914d6c99164e68dc088bd7" candidate_uuid: "98f353c6bf464faaa685c883ab892a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d2869b629741b6a5e760a3c34d39fc" is_pre_election: true
I20260812 03:56:34.655607 7292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395f3886e5914d6c99164e68dc088bd7" candidate_uuid: "98f353c6bf464faaa685c883ab892a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8227dfe9d9684ac3b3618a5d417d172c" is_pre_election: true
I20260812 03:56:34.655771 7292 raft_consensus.cc:2454] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98f353c6bf464faaa685c883ab892a40 in term 0.
I20260812 03:56:34.655769 7448 raft_consensus.cc:2454] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98f353c6bf464faaa685c883ab892a40 in term 0.
I20260812 03:56:34.656036 7526 leader_election.cc:302] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8227dfe9d9684ac3b3618a5d417d172c, 98f353c6bf464faaa685c883ab892a40; no voters:
I20260812 03:56:34.656190 7690 raft_consensus.cc:2781] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:56:34.656260 7690 raft_consensus.cc:482] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:56:34.656277 7690 raft_consensus.cc:3037] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:34.657049 7690 raft_consensus.cc:504] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.657194 7690 leader_election.cc:288] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [CANDIDATE]: Term 1 election: Requested vote from peers 8227dfe9d9684ac3b3618a5d417d172c (127.17.54.193:36427), 22d2869b629741b6a5e760a3c34d39fc (127.17.54.194:45175)
I20260812 03:56:34.657411 7292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395f3886e5914d6c99164e68dc088bd7" candidate_uuid: "98f353c6bf464faaa685c883ab892a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8227dfe9d9684ac3b3618a5d417d172c"
I20260812 03:56:34.657424 7448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395f3886e5914d6c99164e68dc088bd7" candidate_uuid: "98f353c6bf464faaa685c883ab892a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d2869b629741b6a5e760a3c34d39fc"
I20260812 03:56:34.657490 7292 raft_consensus.cc:3037] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:34.657500 7448 raft_consensus.cc:3037] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:34.658191 7292 raft_consensus.cc:2454] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98f353c6bf464faaa685c883ab892a40 in term 1.
I20260812 03:56:34.658365 7448 raft_consensus.cc:2454] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98f353c6bf464faaa685c883ab892a40 in term 1.
I20260812 03:56:34.658406 7526 leader_election.cc:302] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8227dfe9d9684ac3b3618a5d417d172c, 98f353c6bf464faaa685c883ab892a40; no voters:
I20260812 03:56:34.658540 7690 raft_consensus.cc:2781] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:34.658751 7690 raft_consensus.cc:686] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 1 LEADER]: Becoming Leader. State: Replica: 98f353c6bf464faaa685c883ab892a40, State: Running, Role: LEADER
I20260812 03:56:34.658852 7690 consensus_queue.cc:237] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } }
I20260812 03:56:34.659834 7125 catalog_manager.cc:5495] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98f353c6bf464faaa685c883ab892a40 (127.17.54.195). New cstate: current_term: 1 leader_uuid: "98f353c6bf464faaa685c883ab892a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98f353c6bf464faaa685c883ab892a40" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 34623 } health_report { overall_health: HEALTHY } } }
I20260812 03:56:34.666656 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.664244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43162 (local address 127.17.54.195:34623)
0812 03:56:34.664292 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.664295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.664304 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:56:34.664335 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.664335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.664352 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.664422 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:56:34.664611 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.665128 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.665947 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.666142 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.666196 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.666367 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:56:34.666469 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:56:34.666532 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.666570 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.666587 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:56:34.668350 7448 raft_consensus.cc:1261] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 1 FOLLOWER]: Refusing update from remote peer 98f353c6bf464faaa685c883ab892a40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:34.668377 7292 raft_consensus.cc:1261] T 395f3886e5914d6c99164e68dc088bd7 P 8227dfe9d9684ac3b3618a5d417d172c [term 1 FOLLOWER]: Refusing update from remote peer 98f353c6bf464faaa685c883ab892a40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:34.668702 7690 consensus_queue.cc:1035] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d2869b629741b6a5e760a3c34d39fc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 45175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:34.668853 7693 consensus_queue.cc:1035] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8227dfe9d9684ac3b3618a5d417d172c" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 36427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:34.671558 7697 mvcc.cc:205] Tried to move back new op lower bound from 7317532650159112192 to 7317532650123620352. Current Snapshot: MvccSnapshot[applied={T|T < 7317532650159112192}]
I20260812 03:56:34.672236 7696 mvcc.cc:205] Tried to move back new op lower bound from 7317532650159112192 to 7317532650123620352. Current Snapshot: MvccSnapshot[applied={T|T < 7317532650159112192 or (T in {7317532650159112192})}]
I20260812 03:56:34.673991 7125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:34.674098 7125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:34.674937 7125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5212f73a03f2410287aaf7a813a82fa5: Sending DeleteTablet for 3 replicas of tablet 395f3886e5914d6c99164e68dc088bd7
I20260812 03:56:34.675289 7270 tablet_service.cc:1514] Processing DeleteTablet for tablet 395f3886e5914d6c99164e68dc088bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:34 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:47624
I20260812 03:56:34.675324 7587 tablet_service.cc:1514] Processing DeleteTablet for tablet 395f3886e5914d6c99164e68dc088bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:34 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:43148
I20260812 03:56:34.675529 7426 tablet_service.cc:1514] Processing DeleteTablet for tablet 395f3886e5914d6c99164e68dc088bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:34 UTC) from {username='kudu', principal='kudu/127.17.54.254@KRBTEST.COM'} at 127.0.0.1:48900
I20260812 03:56:34.675560 7708 tablet_replica.cc:330] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: stopping tablet replica
I20260812 03:56:34.675653 7708 raft_consensus.cc:2229] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:56:34.675740 7709 tablet_replica.cc:330] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: stopping tablet replica
I20260812 03:56:34.675766 7708 raft_consensus.cc:2258] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:34.675829 7709 raft_consensus.cc:2229] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:34.675937 7709 raft_consensus.cc:2258] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:34.676158 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 7197
I20260812 03:56:34.676230 7708 ts_tablet_manager.cc:1905] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:34.676357 7709 ts_tablet_manager.cc:1905] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:34.678100 7708 ts_tablet_manager.cc:1918] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:34.678166 7708 log.cc:1196] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/395f3886e5914d6c99164e68dc088bd7
I20260812 03:56:34.678480 7708 ts_tablet_manager.cc:1939] T 395f3886e5914d6c99164e68dc088bd7 P 98f353c6bf464faaa685c883ab892a40: Deleting consensus metadata
I20260812 03:56:34.678685 7709 ts_tablet_manager.cc:1918] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:34.678752 7709 log.cc:1196] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/395f3886e5914d6c99164e68dc088bd7
I20260812 03:56:34.678871 7105 catalog_manager.cc:4841] TS 98f353c6bf464faaa685c883ab892a40 (127.17.54.195:34623): tablet 395f3886e5914d6c99164e68dc088bd7 (table test-table [id=485f840cc76a4ab2a27b289e64dad7b7]) successfully deleted
I20260812 03:56:34.679036 7709 ts_tablet_manager.cc:1939] T 395f3886e5914d6c99164e68dc088bd7 P 22d2869b629741b6a5e760a3c34d39fc: Deleting consensus metadata
I20260812 03:56:34.679592 7105 catalog_manager.cc:4841] TS 22d2869b629741b6a5e760a3c34d39fc (127.17.54.194:45175): tablet 395f3886e5914d6c99164e68dc088bd7 (table test-table [id=485f840cc76a4ab2a27b289e64dad7b7]) successfully deleted
W20260812 03:56:34.685490 7105 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:36427 (error 108)
I20260812 03:56:34.685748 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 7352
I20260812 03:56:34.692560 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.692349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:36427 (local address 127.0.0.1:47638)
0812 03:56:34.692441 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:34.692510 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:36427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 03:56:34.692860 7105 catalog_manager.cc:4584] TS 8227dfe9d9684ac3b3618a5d417d172c (127.17.54.193:36427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 395f3886e5914d6c99164e68dc088bd7: Network error: Client connection negotiation failed: client connection to 127.17.54.193:36427: connect: Connection refused (error 111)
I20260812 03:56:34.697443 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 7509
I20260812 03:56:34.705192 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 7087
2026-08-12T03:56:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2246 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T03:56:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:34Z Disabled control of system clock
I20260812 03:56:34.742797 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39267
--webserver_interface=127.17.54.254
--webserver_port=0
--builtin_ntp_servers=127.17.54.212:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:39267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:56:34.831466 7716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:34.831681 7716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:34.831702 7716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:34.832850 7716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:34.832888 7716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:34.832901 7716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:34.832914 7716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:34.832927 7716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:34.834239 7716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:39267
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:34.834524 7716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:34.834775 7716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:34.839704 7724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:34.839774 7722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:34.839699 7721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:34.839931 7716 server_base.cc:1034] running on GCE node
I20260812 03:56:34.840204 7716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:34.840565 7716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:34.841747 7716 hybrid_clock.cc:648] HybridClock initialized: now 1786506994841714 us; error 54 us; skew 500 ppm
I20260812 03:56:34.843295 7716 webserver.cc:460] Webserver started at http://127.17.54.254:37203/ using document root <none> and password file <none>
I20260812 03:56:34.843590 7716 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:34.843667 7716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:34.843827 7716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:34.844930 7716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "b5122f8e0c114245ba0c7e4978e8d188"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "be0442754f2d2661f20a8bde3b83e13d"
server_key_iv: "099cd72b9b83fd0cc888ce82c3bf0207"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.845338 7716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "b5122f8e0c114245ba0c7e4978e8d188"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "be0442754f2d2661f20a8bde3b83e13d"
server_key_iv: "099cd72b9b83fd0cc888ce82c3bf0207"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.846879 7716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260812 03:56:34.847754 7732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:34.847972 7716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:56:34.848089 7716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "b5122f8e0c114245ba0c7e4978e8d188"
format_stamp: "Formatted at 2026-08-12 03:56:34 on dist-test-slave-tm1g"
server_key: "be0442754f2d2661f20a8bde3b83e13d"
server_key_iv: "099cd72b9b83fd0cc888ce82c3bf0207"
server_key_version: "encryptionkey@0"
I20260812 03:56:34.848169 7716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:34.862514 7716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:34.862866 7716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:34.863030 7716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:34.867426 7716 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:39267
I20260812 03:56:34.867461 7788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:39267 every 8 connection(s)
I20260812 03:56:34.867821 7716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:34.868181 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 7716
I20260812 03:56:34.868292 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:34.868494 17627 external_mini_cluster.cc:1428] Setting key 942e685f65070c4bd820a1f411a9cb17
I20260812 03:56:34.869086 7790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:34.872396 7790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188: Bootstrap starting.
I20260812 03:56:34.873121 7790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:34.873447 7790 log.cc:824] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:34.874290 7790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188: No bootstrap required, opened a new log
I20260812 03:56:34.874729 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:34.869085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.17.54.254:39267)
0812 03:56:34.869323 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:34.869332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:34.869364 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:56:34.869902 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:34.869911 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:34.869992 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:34.870145 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:34.870353 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.871721 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.872533 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:34.873139 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:34.873275 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:34.874002 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:34.874006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:34.874007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:34.874210 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:34.874250 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:34.874264 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:34.874363 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:34.874470 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 03:56:34.874555 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260812 03:56:34.876413 7790 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5122f8e0c114245ba0c7e4978e8d188" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39267 } }
I20260812 03:56:34.876544 7790 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:34.876567 7790 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5122f8e0c114245ba0c7e4978e8d188, State: Initialized, Role: FOLLOWER
I20260812 03:56:34.876654 7790 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5122f8e0c114245ba0c7e4978e8d188" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39267 } }
I20260812 03:56:34.876705 7790 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:34.876726 7790 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:34.876752 7790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:34.877259 7790 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5122f8e0c114245ba0c7e4978e8d188" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39267 } }
I20260812 03:56:34.877341 7790 leader_election.cc:302] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5122f8e0c114245ba0c7e4978e8d188; no voters:
I20260812 03:56:34.877497 7790 leader_election.cc:288] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:34.877538 7797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:34.877663 7797 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [term 1 LEADER]: Becoming Leader. State: Replica: b5122f8e0c114245ba0c7e4978e8d188, State: Running, Role: LEADER
I20260812 03:56:34.877774 7790 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:34.877755 7797 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5122f8e0c114245ba0c7e4978e8d188" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39267 } }
I20260812 03:56:34.878096 7797 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5122f8e0c114245ba0c7e4978e8d188. Latest consensus state: current_term: 1 leader_uuid: "b5122f8e0c114245ba0c7e4978e8d188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5122f8e0c114245ba0c7e4978e8d188" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39267 } } }
I20260812 03:56:34.878171 7797 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:34.878521 7803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:34.879094 7803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:34.878126 7799 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5122f8e0c114245ba0c7e4978e8d188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5122f8e0c114245ba0c7e4978e8d188" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39267 } } }
I20260812 03:56:34.879735 7799 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:34.880703 7803 catalog_manager.cc:1347] Generated new cluster ID: bda4c7f6bcf34b2bb42c18614963f7fe
I20260812 03:56:34.880748 7803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:34.918721 7803 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:34.919278 7803 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:34.930923 7803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188: Generated new TSK 0
I20260812 03:56:34.931129 7803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:34.934473 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.193:0
--local_ip_for_outbound_sockets=127.17.54.193
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39267
--builtin_ntp_servers=127.17.54.212:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:56:35.017998 7818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:35.018222 7818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:35.018245 7818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:35.019455 7818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:35.019497 7818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:35.019523 7818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.193
I20260812 03:56:35.020885 7818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.54.193
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.193
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:35.021135 7818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:35.021466 7818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:35.026904 7823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:35.026898 7826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:35.026961 7818 server_base.cc:1034] running on GCE node
W20260812 03:56:35.026902 7824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:35.027328 7818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:35.027587 7818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:35.028827 7818 hybrid_clock.cc:648] HybridClock initialized: now 1786506995028810 us; error 44 us; skew 500 ppm
I20260812 03:56:35.030413 7818 webserver.cc:460] Webserver started at http://127.17.54.193:40141/ using document root <none> and password file <none>
I20260812 03:56:35.030671 7818 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:35.030721 7818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:35.030854 7818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:35.031989 7818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/data/instance:
uuid: "7985a1eaef84419eacadbcacfe0b5b57"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "bdfddd576298a08238b988931e54aa5a"
server_key_iv: "982517211c1b71da7b7a85f9c1514e14"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.032373 7818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance:
uuid: "7985a1eaef84419eacadbcacfe0b5b57"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "bdfddd576298a08238b988931e54aa5a"
server_key_iv: "982517211c1b71da7b7a85f9c1514e14"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.033725 7818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:35.034508 7832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.034749 7818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:35.034879 7818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/wal
uuid: "7985a1eaef84419eacadbcacfe0b5b57"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "bdfddd576298a08238b988931e54aa5a"
server_key_iv: "982517211c1b71da7b7a85f9c1514e14"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.034962 7818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:35.044485 7818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:35.044816 7818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:35.044977 7818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:35.045218 7818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:35.045603 7818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:35.045662 7818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.045708 7818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:35.045733 7818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.052942 7818 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.193:33151
I20260812 03:56:35.052984 7962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.193:33151 every 8 connection(s)
I20260812 03:56:35.053396 7818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/data/info.pb
I20260812 03:56:35.059233 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.054185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:45477 (local address 127.17.54.254:39267)
0812 03:56:35.054234 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.054236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.054579 (+ 343us) server_negotiation.cc:407] Connection header received
0812 03:56:35.055135 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.055137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.055155 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.055214 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:35.055980 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.056539 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.057577 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.057783 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.057863 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.058891 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:35.058893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:35.058893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:35.058899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:35.058917 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:35.058926 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:35.058996 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.059099 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.059143 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:56:35.059461 7966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.054208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:39267 (local address 127.17.54.193:45477)
0812 03:56:35.054451 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.054487 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:35.055025 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:35.055279 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:35.055283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:35.055293 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:35.055864 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.055870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.056641 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.056642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.057478 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.057480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.057883 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.057885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.058022 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.058668 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:35.058669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:35.058700 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:35.058725 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:35.058996 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:35.059004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:35.059008 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:56:35.059102 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:56:35.059216 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":133,"threads_started":1}
I20260812 03:56:35.060024 7963 heartbeater.cc:347] Connected to a master server at 127.17.54.254:39267
I20260812 03:56:35.060139 7963 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:35.060359 7963 heartbeater.cc:511] Master 127.17.54.254:39267 requested a full tablet report, sending...
I20260812 03:56:35.060380 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 7818
I20260812 03:56:35.060504 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-0/wal/instance
I20260812 03:56:35.060729 17627 external_mini_cluster.cc:1428] Setting key 97d7f77d48b28aa81293a2b9347e8070
I20260812 03:56:35.061007 7750 ts_manager.cc:194] Registered new tserver with Master: 7985a1eaef84419eacadbcacfe0b5b57 (127.17.54.193:33151)
I20260812 03:56:35.062016 7750 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.193:45477
I20260812 03:56:35.062086 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.194:0
--local_ip_for_outbound_sockets=127.17.54.194
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39267
--builtin_ntp_servers=127.17.54.212:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:56:35.146260 7967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:35.146617 7967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:35.146643 7967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:35.147796 7967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:35.147835 7967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:35.147859 7967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.194
I20260812 03:56:35.149189 7967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.54.194
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.194
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:35.149400 7967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:35.149665 7967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:35.154978 7976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:35.155056 7974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:35.154978 7973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:35.155224 7967 server_base.cc:1034] running on GCE node
I20260812 03:56:35.155447 7967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:35.155733 7967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:35.156909 7967 hybrid_clock.cc:648] HybridClock initialized: now 1786506995156896 us; error 38 us; skew 500 ppm
I20260812 03:56:35.158221 7967 webserver.cc:460] Webserver started at http://127.17.54.194:38547/ using document root <none> and password file <none>
I20260812 03:56:35.158536 7967 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:35.158614 7967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:35.158775 7967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:35.159883 7967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/data/instance:
uuid: "2d872ab641104eaabd080381d457c4bc"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "a0d7c14a1414aab5efb8e8c80371046b"
server_key_iv: "d75e7b2b7a8068f79f19b6d54ce60c9c"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.160260 7967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance:
uuid: "2d872ab641104eaabd080381d457c4bc"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "a0d7c14a1414aab5efb8e8c80371046b"
server_key_iv: "d75e7b2b7a8068f79f19b6d54ce60c9c"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.161664 7967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:35.162460 7982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.162639 7967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:35.162756 7967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/wal
uuid: "2d872ab641104eaabd080381d457c4bc"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "a0d7c14a1414aab5efb8e8c80371046b"
server_key_iv: "d75e7b2b7a8068f79f19b6d54ce60c9c"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.162844 7967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:35.186143 7967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:35.186545 7967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:35.186712 7967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:35.186969 7967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:35.187328 7967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:35.187387 7967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.187430 7967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:35.187482 7967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.193456 7967 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.194:39281
I20260812 03:56:35.193509 8112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.194:39281 every 8 connection(s)
I20260812 03:56:35.193897 7967 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/data/info.pb
I20260812 03:56:35.198086 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 7967
I20260812 03:56:35.198210 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/wal/instance
I20260812 03:56:35.198470 17627 external_mini_cluster.cc:1428] Setting key 8afdeb603e3e809fc592c2e2295b2e41
I20260812 03:56:35.199348 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.194656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.194:33113 (local address 127.17.54.254:39267)
0812 03:56:35.194722 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.194725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.194967 (+ 242us) server_negotiation.cc:407] Connection header received
0812 03:56:35.195464 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.195465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.195476 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.195529 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:35.196166 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.196667 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.197628 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.197818 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.197892 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.199019 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:35.199020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:35.199021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:35.199025 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:35.199042 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:35.199046 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:35.199106 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.199228 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.199257 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:56:35.199594 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.194647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:39267 (local address 127.17.54.194:33113)
0812 03:56:35.194848 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.194883 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:35.195372 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:35.195581 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:35.195584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:35.195591 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:35.196095 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.196099 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.196749 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.196750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.197546 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.197548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.197928 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.197929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.198113 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.198860 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:35.198861 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:35.198897 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:35.198929 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:35.199129 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:35.199136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:35.199139 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:35.199212 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:56:35.199336 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260812 03:56:35.199621 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.54.195:0
--local_ip_for_outbound_sockets=127.17.54.195
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39267
--builtin_ntp_servers=127.17.54.212:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:56:35.200218 8114 heartbeater.cc:347] Connected to a master server at 127.17.54.254:39267
I20260812 03:56:35.200340 8114 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:35.200528 8114 heartbeater.cc:511] Master 127.17.54.254:39267 requested a full tablet report, sending...
I20260812 03:56:35.200923 7750 ts_manager.cc:194] Registered new tserver with Master: 2d872ab641104eaabd080381d457c4bc (127.17.54.194:39281)
I20260812 03:56:35.201472 7750 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.194:33113
W20260812 03:56:35.284425 8117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:35.284632 8117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:35.284654 8117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:35.285745 8117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:35.285779 8117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:35.285804 8117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.54.195
I20260812 03:56:35.287097 8117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.54.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.54.195
--webserver_port=0
--tserver_master_addrs=127.17.54.254:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.54.195
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:35.287293 8117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:35.287552 8117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:35.292657 8123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:35.292733 8117 server_base.cc:1034] running on GCE node
W20260812 03:56:35.292778 8122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:35.292657 8125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:35.293125 8117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:35.293331 8117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:35.294466 8117 hybrid_clock.cc:648] HybridClock initialized: now 1786506995294453 us; error 28 us; skew 500 ppm
I20260812 03:56:35.295783 8117 webserver.cc:460] Webserver started at http://127.17.54.195:34369/ using document root <none> and password file <none>
I20260812 03:56:35.296031 8117 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:35.296079 8117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:35.296247 8117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:35.297315 8117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/data/instance:
uuid: "84edebdf35224c4ea27c521ce9c9849f"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "821f42d73a5c06819edf404a591ab24a"
server_key_iv: "4291089d073b8cd47bd5badefcc92825"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.297679 8117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance:
uuid: "84edebdf35224c4ea27c521ce9c9849f"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "821f42d73a5c06819edf404a591ab24a"
server_key_iv: "4291089d073b8cd47bd5badefcc92825"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.299089 8117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:35.299968 8133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.300153 8117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:35.300277 8117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/wal
uuid: "84edebdf35224c4ea27c521ce9c9849f"
format_stamp: "Formatted at 2026-08-12 03:56:35 on dist-test-slave-tm1g"
server_key: "821f42d73a5c06819edf404a591ab24a"
server_key_iv: "4291089d073b8cd47bd5badefcc92825"
server_key_version: "encryptionkey@0"
I20260812 03:56:35.300401 8117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:35.331297 8117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:35.331661 8117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:35.331828 8117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:35.332110 8117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:56:35.332474 8117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:56:35.332540 8117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.332584 8117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:56:35.332610 8117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:35.339175 8117 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.195:33805
I20260812 03:56:35.339226 8260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.195:33805 every 8 connection(s)
I20260812 03:56:35.339581 8117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/data/info.pb
I20260812 03:56:35.344762 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.340333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.195:45161 (local address 127.17.54.254:39267)
0812 03:56:35.340395 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.340397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.340626 (+ 229us) server_negotiation.cc:407] Connection header received
0812 03:56:35.341125 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.341126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.341137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.341209 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:35.341926 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.342472 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.343450 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.343643 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.343703 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.344518 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:35.344520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:35.344520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:35.344525 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:35.344538 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:35.344542 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:35.344598 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.344679 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.344703 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:56:35.344949 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.340289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:39267 (local address 127.17.54.195:45161)
0812 03:56:35.340490 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.340533 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:35.341027 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:35.341286 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:35.341290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:35.341298 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:56:35.341814 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.341817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.342563 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.342563 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.343354 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.343356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.343726 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.343727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.343843 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.344368 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:56:35.344369 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:56:35.344396 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:56:35.344419 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:56:35.344606 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:56:35.344610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:56:35.344613 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:35.344665 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:56:35.344744 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260812 03:56:35.345525 8261 heartbeater.cc:347] Connected to a master server at 127.17.54.254:39267
I20260812 03:56:35.345546 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 8117
I20260812 03:56:35.345639 8261 heartbeater.cc:464] Registering TS with master...
I20260812 03:56:35.345669 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/wal/instance
I20260812 03:56:35.345854 8261 heartbeater.cc:511] Master 127.17.54.254:39267 requested a full tablet report, sending...
I20260812 03:56:35.345870 17627 external_mini_cluster.cc:1428] Setting key a83568fd10762cabb4f56a6073309860
I20260812 03:56:35.346269 7749 ts_manager.cc:194] Registered new tserver with Master: 84edebdf35224c4ea27c521ce9c9849f (127.17.54.195:33805)
I20260812 03:56:35.346864 7749 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.54.195:45161
I20260812 03:56:35.347121 17627 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:56:35.351398 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.348544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37618 (local address 127.17.54.254:39267)
0812 03:56:35.348579 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.348581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.348705 (+ 124us) server_negotiation.cc:407] Connection header received
0812 03:56:35.348761 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.348761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.348769 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.348823 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:35.348944 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.349421 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.350172 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.350426 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.350483 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.351144 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:35.351145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:35.351145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:35.351149 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:35.351162 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:35.351165 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:35.351208 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.351251 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.351270 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:56:35.353195 7749 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:56:35.354759 7749 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:56:35.361430 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.357191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:33805 (local address 127.0.0.1:59082)
0812 03:56:35.357609 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.357652 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:35.357743 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:35.358033 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:35.358037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:35.358040 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:35.358108 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.358111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.359166 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.359166 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.360278 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.360281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.361111 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.361112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.361194 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.361208 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:56:35.361287 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:56:35.361337 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260812 03:56:35.361446 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.357401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.17.54.195:33805)
0812 03:56:35.357635 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.357639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.357737 (+ 98us) server_negotiation.cc:407] Connection header received
0812 03:56:35.357817 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.357818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.357831 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.357949 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:35.358191 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.359062 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.360370 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.361011 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.361085 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.361172 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.361316 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.361360 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":128,"threads_started":1}
I20260812 03:56:35.361989 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.357191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:39281 (local address 127.0.0.1:48622)
0812 03:56:35.357842 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.357850 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:35.357918 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:35.358226 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:35.358227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:35.358228 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:35.358277 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.358279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.359696 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.359696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.360651 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.360653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.361776 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.361777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.361832 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.361836 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:56:35.361878 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 03:56:35.361927 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260812 03:56:35.362008 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.357403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.17.54.194:39281)
0812 03:56:35.357590 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.357595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.357999 (+ 404us) server_negotiation.cc:407] Connection header received
0812 03:56:35.358055 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.358057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.358075 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.358149 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:35.358412 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.359587 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.360747 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.361461 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.361772 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.361866 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.361894 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.361929 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260812 03:56:35.362365 8185 tablet_service.cc:1467] Processing CreateTablet for tablet 7433471b46a74ad689eeed1dea4a97e9 (DEFAULT_TABLE table=test-table [id=7043b6f431d945ee832f305e2cedcbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:35.362658 8036 tablet_service.cc:1467] Processing CreateTablet for tablet 7433471b46a74ad689eeed1dea4a97e9 (DEFAULT_TABLE table=test-table [id=7043b6f431d945ee832f305e2cedcbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:35.362784 8185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7433471b46a74ad689eeed1dea4a97e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:35.362999 8036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7433471b46a74ad689eeed1dea4a97e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:35.363222 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.357197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33151 (local address 127.0.0.1:44788)
0812 03:56:35.358019 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.358029 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:35.358075 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:35.358271 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:35.358271 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:35.358273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:35.358334 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.358336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.360723 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.360723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.361636 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.361638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.362921 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.362923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.362989 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.362992 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:56:35.363121 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 03:56:35.363165 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":99,"threads_started":1}
I20260812 03:56:35.363518 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.357472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.17.54.193:33151)
0812 03:56:35.357732 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.357736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.358135 (+ 399us) server_negotiation.cc:407] Connection header received
0812 03:56:35.358180 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.358182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.358197 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.358271 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:35.358460 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.359594 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.361731 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.362758 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.362848 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.362934 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.363386 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.363432 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
I20260812 03:56:35.364184 7890 tablet_service.cc:1467] Processing CreateTablet for tablet 7433471b46a74ad689eeed1dea4a97e9 (DEFAULT_TABLE table=test-table [id=7043b6f431d945ee832f305e2cedcbe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:56:35.364512 7890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7433471b46a74ad689eeed1dea4a97e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:35.366343 8276 tablet_bootstrap.cc:492] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Bootstrap starting.
I20260812 03:56:35.367136 8276 tablet_bootstrap.cc:654] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:35.367504 8276 log.cc:824] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:35.367861 8275 tablet_bootstrap.cc:492] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Bootstrap starting.
I20260812 03:56:35.368229 8276 tablet_bootstrap.cc:492] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: No bootstrap required, opened a new log
I20260812 03:56:35.368315 8276 ts_tablet_manager.cc:1397] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:56:35.368492 8275 tablet_bootstrap.cc:654] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:35.368829 8275 log.cc:824] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:35.369581 8275 tablet_bootstrap.cc:492] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: No bootstrap required, opened a new log
I20260812 03:56:35.369633 8275 ts_tablet_manager.cc:1397] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:56:35.370594 8278 tablet_bootstrap.cc:492] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57: Bootstrap starting.
I20260812 03:56:35.370622 8276 raft_consensus.cc:348] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.370781 8276 raft_consensus.cc:374] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:35.370828 8276 raft_consensus.cc:729] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d872ab641104eaabd080381d457c4bc, State: Initialized, Role: FOLLOWER
I20260812 03:56:35.370981 8276 consensus_queue.cc:260] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.371239 8276 ts_tablet_manager.cc:1428] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:56:35.371240 8278 tablet_bootstrap.cc:654] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:35.371378 8114 heartbeater.cc:503] Master 127.17.54.254:39267 was elected leader, sending a full tablet report...
I20260812 03:56:35.371593 8278 log.cc:824] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:35.372450 8278 tablet_bootstrap.cc:492] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57: No bootstrap required, opened a new log
I20260812 03:56:35.372531 8278 ts_tablet_manager.cc:1397] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:56:35.372514 8275 raft_consensus.cc:348] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.372632 8275 raft_consensus.cc:374] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:35.372686 8275 raft_consensus.cc:729] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84edebdf35224c4ea27c521ce9c9849f, State: Initialized, Role: FOLLOWER
I20260812 03:56:35.372815 8275 consensus_queue.cc:260] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.373068 8275 ts_tablet_manager.cc:1428] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:56:35.373208 8261 heartbeater.cc:503] Master 127.17.54.254:39267 was elected leader, sending a full tablet report...
I20260812 03:56:35.374876 8278 raft_consensus.cc:348] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.375015 8278 raft_consensus.cc:374] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:35.375059 8278 raft_consensus.cc:729] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7985a1eaef84419eacadbcacfe0b5b57, State: Initialized, Role: FOLLOWER
I20260812 03:56:35.375170 8278 consensus_queue.cc:260] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.375414 8278 ts_tablet_manager.cc:1428] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 03:56:35.375468 7963 heartbeater.cc:503] Master 127.17.54.254:39267 was elected leader, sending a full tablet report...
W20260812 03:56:35.444442 8115 tablet.cc:2369] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:56:35.554209 7964 tablet.cc:2369] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:56:35.590163 8262 tablet.cc:2369] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:56:35.649317 8285 raft_consensus.cc:482] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:56:35.649446 8285 raft_consensus.cc:504] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.649873 8285 leader_election.cc:288] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84edebdf35224c4ea27c521ce9c9849f (127.17.54.195:33805), 2d872ab641104eaabd080381d457c4bc (127.17.54.194:39281)
I20260812 03:56:35.653499 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.650073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.194:39281 (local address 127.17.54.193:58601)
0812 03:56:35.650226 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.650238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:35.650377 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:35.650557 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:35.650558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:35.650562 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:35.650685 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.650688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.651664 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.651666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.652602 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.652605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.653190 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.653191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.653270 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.653275 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:56:35.653334 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:56:35.653398 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":77,"thread_start_us":65,"threads_started":1}
I20260812 03:56:35.653471 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.650176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:58601 (local address 127.17.54.194:39281)
0812 03:56:35.650243 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.650245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.650292 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:56:35.650441 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.650442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.650454 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.650542 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:35.650783 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.651546 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.652697 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.653092 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.653151 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.653157 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.653350 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.653399 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:56:35.653541 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.650176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.54.193:36503 (local address 127.17.54.195:33805)
0812 03:56:35.650239 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.650241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.650317 (+ 76us) server_negotiation.cc:407] Connection header received
0812 03:56:35.650415 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.650416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.650428 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.650509 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:56:35.650784 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.651546 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.652770 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.653175 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.653249 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.653259 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.653424 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.653467 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:56:35.653532 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.650072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.195:33805 (local address 127.17.54.193:36503)
0812 03:56:35.650226 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.650238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:56:35.650327 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:56:35.650557 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:56:35.650558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:56:35.650562 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:56:35.650692 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.650693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.651664 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.651666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.652610 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:56:35.652614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.653258 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.653260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:56:35.653335 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.653340 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:56:35.653399 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:56:35.653436 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":73,"thread_start_us":76,"threads_started":1}
I20260812 03:56:35.653890 8061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7433471b46a74ad689eeed1dea4a97e9" candidate_uuid: "7985a1eaef84419eacadbcacfe0b5b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d872ab641104eaabd080381d457c4bc" is_pre_election: true
I20260812 03:56:35.654057 8061 raft_consensus.cc:2454] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7985a1eaef84419eacadbcacfe0b5b57 in term 0.
I20260812 03:56:35.654109 8211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7433471b46a74ad689eeed1dea4a97e9" candidate_uuid: "7985a1eaef84419eacadbcacfe0b5b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84edebdf35224c4ea27c521ce9c9849f" is_pre_election: true
I20260812 03:56:35.654278 8211 raft_consensus.cc:2454] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7985a1eaef84419eacadbcacfe0b5b57 in term 0.
I20260812 03:56:35.654377 7836 leader_election.cc:302] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d872ab641104eaabd080381d457c4bc, 7985a1eaef84419eacadbcacfe0b5b57; no voters:
I20260812 03:56:35.654515 8285 raft_consensus.cc:2781] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:56:35.654587 8285 raft_consensus.cc:482] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:56:35.654620 8285 raft_consensus.cc:3037] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:35.655373 8285 raft_consensus.cc:504] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.655539 8285 leader_election.cc:288] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [CANDIDATE]: Term 1 election: Requested vote from peers 84edebdf35224c4ea27c521ce9c9849f (127.17.54.195:33805), 2d872ab641104eaabd080381d457c4bc (127.17.54.194:39281)
I20260812 03:56:35.655735 8211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7433471b46a74ad689eeed1dea4a97e9" candidate_uuid: "7985a1eaef84419eacadbcacfe0b5b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84edebdf35224c4ea27c521ce9c9849f"
I20260812 03:56:35.655733 8061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7433471b46a74ad689eeed1dea4a97e9" candidate_uuid: "7985a1eaef84419eacadbcacfe0b5b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d872ab641104eaabd080381d457c4bc"
I20260812 03:56:35.655817 8211 raft_consensus.cc:3037] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:35.655817 8061 raft_consensus.cc:3037] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:35.656718 8061 raft_consensus.cc:2454] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7985a1eaef84419eacadbcacfe0b5b57 in term 1.
I20260812 03:56:35.656718 8211 raft_consensus.cc:2454] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7985a1eaef84419eacadbcacfe0b5b57 in term 1.
I20260812 03:56:35.656903 7836 leader_election.cc:302] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d872ab641104eaabd080381d457c4bc, 7985a1eaef84419eacadbcacfe0b5b57; no voters:
I20260812 03:56:35.657042 8285 raft_consensus.cc:2781] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:35.657179 8285 raft_consensus.cc:686] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [term 1 LEADER]: Becoming Leader. State: Replica: 7985a1eaef84419eacadbcacfe0b5b57, State: Running, Role: LEADER
I20260812 03:56:35.657276 8285 consensus_queue.cc:237] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } }
I20260812 03:56:35.658092 7750 catalog_manager.cc:5495] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7985a1eaef84419eacadbcacfe0b5b57 (127.17.54.193). New cstate: current_term: 1 leader_uuid: "7985a1eaef84419eacadbcacfe0b5b57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7985a1eaef84419eacadbcacfe0b5b57" member_type: VOTER last_known_addr { host: "127.17.54.193" port: 33151 } health_report { overall_health: HEALTHY } } }
I20260812 03:56:35.714172 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.711957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.17.54.193:33151)
0812 03:56:35.712037 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:35.712039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:35.712047 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:56:35.712086 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:35.712087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:35.712108 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:35.712170 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:56:35.712277 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.712747 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.713469 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:35.713652 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:35.713703 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:35.713840 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:56:35.713938 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:56:35.714004 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:35.714035 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 03:56:35.714056 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:56:35.716142 8211 raft_consensus.cc:1261] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 1 FOLLOWER]: Refusing update from remote peer 7985a1eaef84419eacadbcacfe0b5b57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:35.716162 8061 raft_consensus.cc:1261] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 1 FOLLOWER]: Refusing update from remote peer 7985a1eaef84419eacadbcacfe0b5b57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:56:35.716534 8288 consensus_queue.cc:1035] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d872ab641104eaabd080381d457c4bc" member_type: VOTER last_known_addr { host: "127.17.54.194" port: 39281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:35.716656 8285 consensus_queue.cc:1035] T 7433471b46a74ad689eeed1dea4a97e9 P 7985a1eaef84419eacadbcacfe0b5b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84edebdf35224c4ea27c521ce9c9849f" member_type: VOTER last_known_addr { host: "127.17.54.195" port: 33805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:56:35.719012 8291 mvcc.cc:205] Tried to move back new op lower bound from 7317532654450520064 to 7317532654212796416. Current Snapshot: MvccSnapshot[applied={T|T < 7317532654450520064}]
I20260812 03:56:35.719342 8293 mvcc.cc:205] Tried to move back new op lower bound from 7317532654450520064 to 7317532654212796416. Current Snapshot: MvccSnapshot[applied={T|T < 7317532654450520064}]
I20260812 03:56:35.721478 7750 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:35.721585 7750 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:56:35.722457 7750 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5122f8e0c114245ba0c7e4978e8d188: Sending DeleteTablet for 3 replicas of tablet 7433471b46a74ad689eeed1dea4a97e9
I20260812 03:56:35.722834 8185 tablet_service.cc:1514] Processing DeleteTablet for tablet 7433471b46a74ad689eeed1dea4a97e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:35 UTC) from {username='slave'} at 127.0.0.1:59082
I20260812 03:56:35.722893 7890 tablet_service.cc:1514] Processing DeleteTablet for tablet 7433471b46a74ad689eeed1dea4a97e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:35 UTC) from {username='slave'} at 127.0.0.1:44788
I20260812 03:56:35.723063 8036 tablet_service.cc:1514] Processing DeleteTablet for tablet 7433471b46a74ad689eeed1dea4a97e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:56:35 UTC) from {username='slave'} at 127.0.0.1:48622
I20260812 03:56:35.723110 8304 tablet_replica.cc:330] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: stopping tablet replica
I20260812 03:56:35.723217 8304 raft_consensus.cc:2229] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:35.723327 8304 raft_consensus.cc:2258] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:35.723331 8306 tablet_replica.cc:330] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: stopping tablet replica
I20260812 03:56:35.723510 8306 raft_consensus.cc:2229] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:56:35.723605 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 7818
I20260812 03:56:35.723708 8306 raft_consensus.cc:2258] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:56:35.723860 8304 ts_tablet_manager.cc:1905] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:35.724435 8306 ts_tablet_manager.cc:1905] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:56:35.725876 8304 ts_tablet_manager.cc:1918] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:35.725960 8304 log.cc:1196] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-2/wal/wals/7433471b46a74ad689eeed1dea4a97e9
I20260812 03:56:35.726279 8304 ts_tablet_manager.cc:1939] T 7433471b46a74ad689eeed1dea4a97e9 P 84edebdf35224c4ea27c521ce9c9849f: Deleting consensus metadata
I20260812 03:56:35.726707 8306 ts_tablet_manager.cc:1918] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:56:35.726748 7736 catalog_manager.cc:4841] TS 84edebdf35224c4ea27c521ce9c9849f (127.17.54.195:33805): tablet 7433471b46a74ad689eeed1dea4a97e9 (table test-table [id=7043b6f431d945ee832f305e2cedcbe7]) successfully deleted
I20260812 03:56:35.726907 8306 log.cc:1196] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Deleting WAL directory at /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506741478267-17627-0/minicluster-data/ts-1/wal/wals/7433471b46a74ad689eeed1dea4a97e9
I20260812 03:56:35.727288 8306 ts_tablet_manager.cc:1939] T 7433471b46a74ad689eeed1dea4a97e9 P 2d872ab641104eaabd080381d457c4bc: Deleting consensus metadata
I20260812 03:56:35.727756 7735 catalog_manager.cc:4841] TS 2d872ab641104eaabd080381d457c4bc (127.17.54.194:39281): tablet 7433471b46a74ad689eeed1dea4a97e9 (table test-table [id=7043b6f431d945ee832f305e2cedcbe7]) successfully deleted
W20260812 03:56:35.732941 7733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.54.193:33151 (error 108)
I20260812 03:56:35.733173 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 7967
I20260812 03:56:35.735394 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:35.735232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.193:33151 (local address 127.0.0.1:44818)
0812 03:56:35.735297 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:56:35.735352 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:56:35.735505 7733 catalog_manager.cc:4584] TS 7985a1eaef84419eacadbcacfe0b5b57 (127.17.54.193:33151): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7433471b46a74ad689eeed1dea4a97e9: Network error: Client connection negotiation failed: client connection to 127.17.54.193:33151: connect: Connection refused (error 111)
I20260812 03:56:35.739123 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 8117
I20260812 03:56:35.747239 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 7716
2026-08-12T03:56:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1035 ms)
[----------] 6 tests from EncryptionPolicyTest (9487 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:35 dist-test-slave-tm1g krb5kdc[8311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:35 dist-test-slave-tm1g krb5kdc[8311](info): set up 2 sockets
Aug 12 03:56:35 dist-test-slave-tm1g krb5kdc[8311](info): commencing operation
krb5kdc: starting...
W20260812 03:56:36.859632 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:36 dist-test-slave-tm1g krb5kdc[8311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:36.989851 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44285
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:37.072760 8324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:37.073022 8324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:37.073060 8324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:37.074214 8324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:37.074290 8324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:37.074313 8324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:37.074373 8324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:37.074402 8324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:37.075930 8324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44285
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:37.076218 8324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:37.076450 8324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:37.081605 8334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:37.081643 8330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:37.081605 8332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:37.081714 8324 server_base.cc:1034] running on GCE node
I20260812 03:56:37.082109 8324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:37.082442 8324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:37.083604 8324 hybrid_clock.cc:648] HybridClock initialized: now 1786506997083579 us; error 42 us; skew 500 ppm
Aug 12 03:56:37 dist-test-slave-tm1g krb5kdc[8311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506997, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:37.090200 8324 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:37.090673 8324 webserver.cc:460] Webserver started at http://127.0.0.1:44377/ using document root <none> and password file <none>
I20260812 03:56:37.090865 8324 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:37.090905 8324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:37.090997 8324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:37.092028 8324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "fbec96a928a44412a701f8259773d8ad"
format_stamp: "Formatted at 2026-08-12 03:56:37 on dist-test-slave-tm1g"
server_key: "d78060b34e7a93835de34a927aba94dc"
server_key_iv: "020b7b76e8f3c1fbefdbe69d122d4f9e"
server_key_version: "encryptionkey@0"
I20260812 03:56:37.092335 8324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "fbec96a928a44412a701f8259773d8ad"
format_stamp: "Formatted at 2026-08-12 03:56:37 on dist-test-slave-tm1g"
server_key: "d78060b34e7a93835de34a927aba94dc"
server_key_iv: "020b7b76e8f3c1fbefdbe69d122d4f9e"
server_key_version: "encryptionkey@0"
I20260812 03:56:37.093644 8324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:37.094470 8342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:37.094640 8324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:56:37.094731 8324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "fbec96a928a44412a701f8259773d8ad"
format_stamp: "Formatted at 2026-08-12 03:56:37 on dist-test-slave-tm1g"
server_key: "d78060b34e7a93835de34a927aba94dc"
server_key_iv: "020b7b76e8f3c1fbefdbe69d122d4f9e"
server_key_version: "encryptionkey@0"
I20260812 03:56:37.094790 8324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:37.123431 8324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:37.125955 8324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:37.126081 8324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:37.130123 8324 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44285
I20260812 03:56:37.130191 8403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44285 every 8 connection(s)
I20260812 03:56:37.130564 8324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:37.131486 8404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:37.133741 8404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad: Bootstrap starting.
I20260812 03:56:37.134289 8404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:37.134626 8404 log.cc:824] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:37.135337 8404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad: No bootstrap required, opened a new log
I20260812 03:56:37.136037 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 8324
I20260812 03:56:37.136147 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:37.136394 17627 external_mini_cluster.cc:1428] Setting key fdaa4a996450b9a977c960b85090bef6
I20260812 03:56:37.137483 8404 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbec96a928a44412a701f8259773d8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } }
I20260812 03:56:37.137673 8404 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:37.137720 8404 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbec96a928a44412a701f8259773d8ad, State: Initialized, Role: FOLLOWER
I20260812 03:56:37.137867 8404 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbec96a928a44412a701f8259773d8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } }
I20260812 03:56:37.137960 8404 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:37.137997 8404 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:37.138051 8404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:37.139119 8404 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbec96a928a44412a701f8259773d8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } }
I20260812 03:56:37.139212 8404 leader_election.cc:302] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbec96a928a44412a701f8259773d8ad; no voters:
I20260812 03:56:37.139432 8404 leader_election.cc:288] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:37.139453 8410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:37.139572 8410 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [term 1 LEADER]: Becoming Leader. State: Replica: fbec96a928a44412a701f8259773d8ad, State: Running, Role: LEADER
I20260812 03:56:37.139712 8404 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:37.139720 8410 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbec96a928a44412a701f8259773d8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } }
I20260812 03:56:37.140317 8411 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbec96a928a44412a701f8259773d8ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbec96a928a44412a701f8259773d8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } } }
I20260812 03:56:37.140344 8413 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbec96a928a44412a701f8259773d8ad. Latest consensus state: current_term: 1 leader_uuid: "fbec96a928a44412a701f8259773d8ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbec96a928a44412a701f8259773d8ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44285 } } }
I20260812 03:56:37.140550 8413 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:37.140434 8411 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:37.141418 8422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:37.141979 8422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:56:37 dist-test-slave-tm1g krb5kdc[8311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:56:37.143774 8422 catalog_manager.cc:1347] Generated new cluster ID: 89aa83f2bed343f4934fb42534cd57d5
I20260812 03:56:37.143829 8422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:37.147562 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:37.137086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45571 (local address 127.0.0.1:44285)
0812 03:56:37.137273 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:37.137280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:37.137306 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:56:37.137434 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:37.137439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:37.137483 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:37.137599 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:37.138493 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:37.139621 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:37.140750 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:37.141369 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:37.141520 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:37.144244 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:37.144248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:37.144248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:37.144260 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:37.145949 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:37.146381 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:37.146383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:37.146384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:37.146486 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:37.146734 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:37.146735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:37.146736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:37.146975 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:37.147177 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:37.147272 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 03:56:37.147399 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260812 03:56:37.148691 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:37.156699 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:37.150133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46107 (local address 127.0.0.1:44285)
0812 03:56:37.150174 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:37.150176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:37.150462 (+ 286us) server_negotiation.cc:407] Connection header received
0812 03:56:37.150536 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:37.150537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:37.150547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:37.150601 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:37.151135 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:37.151744 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:37.152451 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:37.152700 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:37.152781 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:37.153995 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:37.153996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:37.153997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:37.154004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:37.155475 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:37.155849 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:37.155851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:37.155851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:37.155919 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:37.156144 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:37.156145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:37.156146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:37.156262 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:37.156391 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:37.156511 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 03:56:37.156575 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":21}
I20260812 03:56:37.172700 8422 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:37.173456 8422 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:37.179023 8422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbec96a928a44412a701f8259773d8ad: Generated new TSK 0
I20260812 03:56:37.179203 8422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:37.185850 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 8324
W20260812 03:56:37.194679 8308 connection.cc:535] client connection to 127.0.0.1:44285 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44285): Connection reset by peer (error 104)
2026-08-12T03:56:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:37 dist-test-slave-tm1g krb5kdc[8440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:37 dist-test-slave-tm1g krb5kdc[8440](info): set up 2 sockets
Aug 12 03:56:37 dist-test-slave-tm1g krb5kdc[8440](info): commencing operation
krb5kdc: starting...
W20260812 03:56:38.300810 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:38 dist-test-slave-tm1g krb5kdc[8440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:38.433691 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:38.521679 8455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:38.521904 8455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:38.521929 8455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:38.523204 8455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:38.523253 8455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:38.523267 8455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:38.523280 8455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:38.523334 8455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:38.524866 8455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33231
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:38.525143 8455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:38.525405 8455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:38.530700 8462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:38.530665 8466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:38.530678 8463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:38.530716 8455 server_base.cc:1034] running on GCE node
I20260812 03:56:38.531157 8455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:38.531514 8455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:38.532701 8455 hybrid_clock.cc:648] HybridClock initialized: now 1786506998532700 us; error 44 us; skew 500 ppm
Aug 12 03:56:38 dist-test-slave-tm1g krb5kdc[8440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:38.535312 8455 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:38.535840 8455 webserver.cc:460] Webserver started at http://127.0.0.1:39511/ using document root <none> and password file <none>
I20260812 03:56:38.536089 8455 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:38.536162 8455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:38.536311 8455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:38.537428 8455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "5e23353bc0be4ab7b74442f49c9b74f8"
format_stamp: "Formatted at 2026-08-12 03:56:38 on dist-test-slave-tm1g"
server_key: "24f33eb08731756bbdaa86e9ac8eb1f7"
server_key_iv: "2b5e4e157e3030ad4b039c97acc61344"
server_key_version: "encryptionkey@0"
I20260812 03:56:38.537808 8455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "5e23353bc0be4ab7b74442f49c9b74f8"
format_stamp: "Formatted at 2026-08-12 03:56:38 on dist-test-slave-tm1g"
server_key: "24f33eb08731756bbdaa86e9ac8eb1f7"
server_key_iv: "2b5e4e157e3030ad4b039c97acc61344"
server_key_version: "encryptionkey@0"
I20260812 03:56:38.539296 8455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:38.540057 8475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:38.540251 8455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:56:38.540354 8455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "5e23353bc0be4ab7b74442f49c9b74f8"
format_stamp: "Formatted at 2026-08-12 03:56:38 on dist-test-slave-tm1g"
server_key: "24f33eb08731756bbdaa86e9ac8eb1f7"
server_key_iv: "2b5e4e157e3030ad4b039c97acc61344"
server_key_version: "encryptionkey@0"
I20260812 03:56:38.540458 8455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:38.551793 8455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:38.553812 8455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:38.553980 8455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:38.558688 8455 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33231
I20260812 03:56:38.558719 8540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33231 every 8 connection(s)
I20260812 03:56:38.559149 8455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:38.559638 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 8455
I20260812 03:56:38.559785 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:38.560019 17627 external_mini_cluster.cc:1428] Setting key 0ed9149aad1b5f419780acc386a49bdd
I20260812 03:56:38.560374 8541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:38.563369 8541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8: Bootstrap starting.
I20260812 03:56:38.564055 8541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:38.564356 8541 log.cc:824] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:38.565238 8541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8: No bootstrap required, opened a new log
I20260812 03:56:38.566030 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:38.560517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37627 (local address 127.0.0.1:33231)
0812 03:56:38.560730 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:38.560740 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:38.560773 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:56:38.561556 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:38.561564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:38.561615 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:38.561781 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:38.562027 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:38.563027 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:38.563898 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:38.564371 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:38.564502 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:38.565247 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:38.565251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:38.565251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:38.565508 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:38.565540 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:38.565552 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:38.565622 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:38.565782 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:56:38.565816 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260812 03:56:38.567202 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:38.567653 8541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e23353bc0be4ab7b74442f49c9b74f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } }
I20260812 03:56:38.567788 8541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:38.567811 8541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e23353bc0be4ab7b74442f49c9b74f8, State: Initialized, Role: FOLLOWER
I20260812 03:56:38.567947 8541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e23353bc0be4ab7b74442f49c9b74f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } }
I20260812 03:56:38.568017 8541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:38.568038 8541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:38.568106 8541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:38.568763 8541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e23353bc0be4ab7b74442f49c9b74f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } }
I20260812 03:56:38.568881 8541 leader_election.cc:302] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e23353bc0be4ab7b74442f49c9b74f8; no voters:
I20260812 03:56:38.569056 8541 leader_election.cc:288] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:38.569098 8552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:38.569290 8552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [term 1 LEADER]: Becoming Leader. State: Replica: 5e23353bc0be4ab7b74442f49c9b74f8, State: Running, Role: LEADER
I20260812 03:56:38.569315 8541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:38.569451 8552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e23353bc0be4ab7b74442f49c9b74f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } }
I20260812 03:56:38.569805 8555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e23353bc0be4ab7b74442f49c9b74f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e23353bc0be4ab7b74442f49c9b74f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } } }
I20260812 03:56:38.569880 8555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:38.570124 8561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:38.570770 8553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e23353bc0be4ab7b74442f49c9b74f8. Latest consensus state: current_term: 1 leader_uuid: "5e23353bc0be4ab7b74442f49c9b74f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e23353bc0be4ab7b74442f49c9b74f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } } }
I20260812 03:56:38.570860 8561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:38.570864 8553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:38.572640 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:38.568773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42235 (local address 127.0.0.1:33231)
0812 03:56:38.568812 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:38.568814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:38.568955 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:56:38.568980 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:38.568981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:38.568992 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:38.569047 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:38.569214 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:38.569975 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:38.571155 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:38.571410 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:38.571465 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:38.572282 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:38.572283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:38.572283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:38.572289 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:38.572306 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:38.572311 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:38.572399 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:38.572478 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 03:56:38.572488 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:56:38.572647 8561 catalog_manager.cc:1347] Generated new cluster ID: 31621071f2294f098cd44bdd1fdca9a5
I20260812 03:56:38.572708 8561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:38.583629 8561 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:38.584249 8561 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:38.588876 8561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e23353bc0be4ab7b74442f49c9b74f8: Generated new TSK 0
I20260812 03:56:38.589104 8561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:38.598580 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 8455
W20260812 03:56:38.606815 8438 connection.cc:535] client connection to 127.0.0.1:33231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33231): Connection reset by peer (error 104)
2026-08-12T03:56:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:38 dist-test-slave-tm1g krb5kdc[8574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:38 dist-test-slave-tm1g krb5kdc[8574](info): set up 2 sockets
Aug 12 03:56:38 dist-test-slave-tm1g krb5kdc[8574](info): commencing operation
krb5kdc: starting...
W20260812 03:56:39.719058 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:39 dist-test-slave-tm1g krb5kdc[8574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:39.847579 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42871
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:39.931568 8591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:39.931859 8591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:39.931900 8591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:39.933216 8591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:39.933296 8591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:39.933321 8591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:39.933341 8591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:39.933369 8591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:39.935035 8591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42871
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:39.935345 8591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:39.935599 8591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:39.940848 8597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:39.940836 8601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:39.940838 8598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:39.941195 8591 server_base.cc:1034] running on GCE node
I20260812 03:56:39.941435 8591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:39.941769 8591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:39.942940 8591 hybrid_clock.cc:648] HybridClock initialized: now 1786506999942914 us; error 39 us; skew 500 ppm
Aug 12 03:56:39 dist-test-slave-tm1g krb5kdc[8574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:39.945647 8591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:39.946045 8591 webserver.cc:460] Webserver started at http://127.0.0.1:33585/ using document root <none> and password file <none>
I20260812 03:56:39.946239 8591 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:39.946280 8591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:39.946444 8591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:39.947522 8591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8"
format_stamp: "Formatted at 2026-08-12 03:56:39 on dist-test-slave-tm1g"
server_key: "ab80f5ae611254b113547133b0396394"
server_key_iv: "9b5bb97f568ef315c006968f593d67c7"
server_key_version: "encryptionkey@0"
I20260812 03:56:39.947875 8591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8"
format_stamp: "Formatted at 2026-08-12 03:56:39 on dist-test-slave-tm1g"
server_key: "ab80f5ae611254b113547133b0396394"
server_key_iv: "9b5bb97f568ef315c006968f593d67c7"
server_key_version: "encryptionkey@0"
I20260812 03:56:39.949210 8591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:39.949924 8609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:39.950109 8591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:56:39.950214 8591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8"
format_stamp: "Formatted at 2026-08-12 03:56:39 on dist-test-slave-tm1g"
server_key: "ab80f5ae611254b113547133b0396394"
server_key_iv: "9b5bb97f568ef315c006968f593d67c7"
server_key_version: "encryptionkey@0"
I20260812 03:56:39.950301 8591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:39.953750 8591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:39.955856 8591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:39.956039 8591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:39.960490 8591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42871
I20260812 03:56:39.960523 8667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42871 every 8 connection(s)
I20260812 03:56:39.960873 8591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:39.961856 8668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:39.963014 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 8591
I20260812 03:56:39.963145 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:39.963318 17627 external_mini_cluster.cc:1428] Setting key 81aadf844b387e9b397e5b199a1349be
I20260812 03:56:39.964879 8668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8: Bootstrap starting.
I20260812 03:56:39.965616 8668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:39.965977 8668 log.cc:824] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:39.966063 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:39.963904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43051 (local address 127.0.0.1:42871)
0812 03:56:39.964108 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:39.964117 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:39.964154 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:56:39.964675 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:39.964684 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:39.964717 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:39.964879 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:39.965588 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:39.965592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:39.965592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:39.965598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:39.965645 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:39.965660 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:39.965760 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:39.965810 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 03:56:39.965854 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260812 03:56:39.966809 8668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8: No bootstrap required, opened a new log
I20260812 03:56:39.967137 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:39.969082 8668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42871 } }
I20260812 03:56:39.969260 8668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:39.969319 8668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05fdee4e91db4b9dbe1b29fe235b2ff8, State: Initialized, Role: FOLLOWER
I20260812 03:56:39.969501 8668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42871 } }
I20260812 03:56:39.969617 8668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:39.969698 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:39.968504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49123 (local address 127.0.0.1:42871)
0812 03:56:39.968538 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:39.968540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:39.968691 (+ 151us) server_negotiation.cc:407] Connection header received
0812 03:56:39.968764 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:39.968765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:39.968770 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:39.968827 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:39.969432 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:39.969433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:39.969434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:39.969438 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:39.969454 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:39.969459 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:39.969520 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:39.969573 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 03:56:39.969588 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:56:39.969710 8668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:39.969792 8668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:39.970468 8668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42871 } }
I20260812 03:56:39.970577 8668 leader_election.cc:302] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05fdee4e91db4b9dbe1b29fe235b2ff8; no voters:
I20260812 03:56:39.970700 8668 leader_election.cc:288] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:39.970762 8680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:39.970916 8680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [term 1 LEADER]: Becoming Leader. State: Replica: 05fdee4e91db4b9dbe1b29fe235b2ff8, State: Running, Role: LEADER
I20260812 03:56:39.970973 8668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:39.971027 8680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42871 } }
I20260812 03:56:39.971383 8681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42871 } } }
I20260812 03:56:39.971515 8681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:39.971432 8683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05fdee4e91db4b9dbe1b29fe235b2ff8. Latest consensus state: current_term: 1 leader_uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fdee4e91db4b9dbe1b29fe235b2ff8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42871 } } }
I20260812 03:56:39.971678 8683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:39.971769 8688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:39.972254 8688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:39.973780 8688 catalog_manager.cc:1347] Generated new cluster ID: 2ddd116da2904ef68ad4b63879eabdcf
I20260812 03:56:39.973843 8688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:39.981361 8688 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:39.981566 8688 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:39.986517 8688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05fdee4e91db4b9dbe1b29fe235b2ff8: Generated new TSK 0
I20260812 03:56:39.986701 8688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:39.989486 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 8591
2026-08-12T03:56:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:40 dist-test-slave-tm1g krb5kdc[8703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:40 dist-test-slave-tm1g krb5kdc[8703](info): set up 2 sockets
Aug 12 03:56:40 dist-test-slave-tm1g krb5kdc[8703](info): commencing operation
krb5kdc: starting...
W20260812 03:56:41.162261 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:41 dist-test-slave-tm1g krb5kdc[8703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:41.289753 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:41.372903 8719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:41.373126 8719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:41.373149 8719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:41.374254 8719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:41.374291 8719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:41.374333 8719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:41.374346 8719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:41.374358 8719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:41.375779 8719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:41.375981 8719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:41.376168 8719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:41.381218 8729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:41.381285 8726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:41.381222 8725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:41.381374 8719 server_base.cc:1034] running on GCE node
I20260812 03:56:41.381750 8719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:41.382048 8719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:41.383198 8719 hybrid_clock.cc:648] HybridClock initialized: now 1786507001383183 us; error 48 us; skew 500 ppm
Aug 12 03:56:41 dist-test-slave-tm1g krb5kdc[8703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:41.385792 8719 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:41.386276 8719 webserver.cc:460] Webserver started at http://127.0.0.1:37063/ using document root <none> and password file <none>
I20260812 03:56:41.386560 8719 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:41.386605 8719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:41.386698 8719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:41.387724 8719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "fcd7c02f4aef450293fcdd7296b08f64"
format_stamp: "Formatted at 2026-08-12 03:56:41 on dist-test-slave-tm1g"
server_key: "a56c980900420e7666c20434c27bb5fc"
server_key_iv: "853f7f60b47ae991d7e1f184804f8e19"
server_key_version: "encryptionkey@0"
I20260812 03:56:41.388042 8719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "fcd7c02f4aef450293fcdd7296b08f64"
format_stamp: "Formatted at 2026-08-12 03:56:41 on dist-test-slave-tm1g"
server_key: "a56c980900420e7666c20434c27bb5fc"
server_key_iv: "853f7f60b47ae991d7e1f184804f8e19"
server_key_version: "encryptionkey@0"
I20260812 03:56:41.389348 8719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:41.390102 8738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:41.390290 8719 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:41.390442 8719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "fcd7c02f4aef450293fcdd7296b08f64"
format_stamp: "Formatted at 2026-08-12 03:56:41 on dist-test-slave-tm1g"
server_key: "a56c980900420e7666c20434c27bb5fc"
server_key_iv: "853f7f60b47ae991d7e1f184804f8e19"
server_key_version: "encryptionkey@0"
I20260812 03:56:41.390504 8719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:41.411152 8719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:41.413484 8719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:41.413602 8719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:41.418294 8719 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34823
I20260812 03:56:41.418802 8719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:41.419374 8795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34823 every 8 connection(s)
I20260812 03:56:41.420459 8796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:41.423022 8796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64: Bootstrap starting.
I20260812 03:56:41.423633 8796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:41.423947 8796 log.cc:824] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:41.424670 8796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64: No bootstrap required, opened a new log
I20260812 03:56:41.425556 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 8719
I20260812 03:56:41.425680 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:41.425887 17627 external_mini_cluster.cc:1428] Setting key 8f46b2232a68245c4ce82e1ee8519fd6
I20260812 03:56:41.426738 8796 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd7c02f4aef450293fcdd7296b08f64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34823 } }
I20260812 03:56:41.426896 8796 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:41.426944 8796 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcd7c02f4aef450293fcdd7296b08f64, State: Initialized, Role: FOLLOWER
I20260812 03:56:41.427147 8796 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd7c02f4aef450293fcdd7296b08f64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34823 } }
I20260812 03:56:41.427250 8796 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:41.427284 8796 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:41.427330 8796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:41.428393 8796 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd7c02f4aef450293fcdd7296b08f64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34823 } }
I20260812 03:56:41.428491 8796 leader_election.cc:302] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcd7c02f4aef450293fcdd7296b08f64; no voters:
I20260812 03:56:41.428676 8796 leader_election.cc:288] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:41.428727 8804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:41.428881 8804 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [term 1 LEADER]: Becoming Leader. State: Replica: fcd7c02f4aef450293fcdd7296b08f64, State: Running, Role: LEADER
I20260812 03:56:41.428978 8796 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:41.429023 8804 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd7c02f4aef450293fcdd7296b08f64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34823 } }
I20260812 03:56:41.429347 8804 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcd7c02f4aef450293fcdd7296b08f64. Latest consensus state: current_term: 1 leader_uuid: "fcd7c02f4aef450293fcdd7296b08f64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd7c02f4aef450293fcdd7296b08f64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34823 } } }
I20260812 03:56:41.429466 8804 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:41.429765 8812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:41.429356 8808 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcd7c02f4aef450293fcdd7296b08f64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd7c02f4aef450293fcdd7296b08f64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34823 } } }
I20260812 03:56:41.430644 8812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:41.432040 8808 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:41.432668 8812 catalog_manager.cc:1347] Generated new cluster ID: 4e9008edf33b46469ce3591b6cf1ba54
I20260812 03:56:41.432716 8812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:56:41 dist-test-slave-tm1g krb5kdc[8703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:56:41.437188 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:41.426504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59413 (local address 127.0.0.1:34823)
0812 03:56:41.426714 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:41.426722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:41.426750 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:56:41.426844 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:41.426850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:41.426891 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:41.427044 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:41.427959 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:41.429309 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:41.430564 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:41.431080 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:41.431212 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:41.433666 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:41.433670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:41.433671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:41.433682 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:41.435441 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:41.435826 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:41.435829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:41.435830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:41.435953 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:41.436168 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:41.436169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:41.436169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:41.436414 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:41.436598 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:41.436666 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 03:56:41.436824 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260812 03:56:41.438184 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:41.449047 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:41.442723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52563 (local address 127.0.0.1:34823)
0812 03:56:41.442779 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:41.442781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:41.443017 (+ 236us) server_negotiation.cc:407] Connection header received
0812 03:56:41.443083 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:41.443084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:41.443094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:41.443144 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:41.443676 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:41.444191 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:41.444897 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:41.445134 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:41.445210 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:41.446490 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:41.446491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:41.446491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:41.446498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:41.447906 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:41.448263 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:41.448265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:41.448265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:41.448322 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:41.448536 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:41.448537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:41.448537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:41.448651 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:41.448778 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:41.448861 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:56:41.448937 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:56:41.449383 8812 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:41.450109 8812 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:41.456195 8812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcd7c02f4aef450293fcdd7296b08f64: Generated new TSK 0
I20260812 03:56:41.456373 8812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:41.465835 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 8719
2026-08-12T03:56:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:41 dist-test-slave-tm1g krb5kdc[8837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:41 dist-test-slave-tm1g krb5kdc[8837](info): set up 2 sockets
Aug 12 03:56:41 dist-test-slave-tm1g krb5kdc[8837](info): commencing operation
krb5kdc: starting...
W20260812 03:56:42.589193 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:42 dist-test-slave-tm1g krb5kdc[8837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:42.721553 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:42.806391 8851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:42.806648 8851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:42.806687 8851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:42.807854 8851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:42.807932 8851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:42.807958 8851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:42.807988 8851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:42.808020 8851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:42.809540 8851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:42.809815 8851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:42.810052 8851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:42.815193 8858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:42.815260 8857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:42.815198 8861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:42.815424 8851 server_base.cc:1034] running on GCE node
I20260812 03:56:42.815735 8851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:42.816025 8851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:42.817233 8851 hybrid_clock.cc:648] HybridClock initialized: now 1786507002817229 us; error 41 us; skew 500 ppm
Aug 12 03:56:42 dist-test-slave-tm1g krb5kdc[8837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507002, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:42.826799 8851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:42.827373 8851 webserver.cc:460] Webserver started at http://127.0.0.1:41985/ using document root <none> and password file <none>
I20260812 03:56:42.827625 8851 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:42.827697 8851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:42.827862 8851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:42.828996 8851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "4b0837fd65fb4bd3a1a3f6e827d27395"
format_stamp: "Formatted at 2026-08-12 03:56:42 on dist-test-slave-tm1g"
server_key: "009f300348a0b04041e2bab637b34f1d"
server_key_iv: "5a03da1aca609e921007d0bcc3217ac8"
server_key_version: "encryptionkey@0"
I20260812 03:56:42.829388 8851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "4b0837fd65fb4bd3a1a3f6e827d27395"
format_stamp: "Formatted at 2026-08-12 03:56:42 on dist-test-slave-tm1g"
server_key: "009f300348a0b04041e2bab637b34f1d"
server_key_iv: "5a03da1aca609e921007d0bcc3217ac8"
server_key_version: "encryptionkey@0"
I20260812 03:56:42.830996 8851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:42.831899 8868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:42.832149 8851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:42.832267 8851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "4b0837fd65fb4bd3a1a3f6e827d27395"
format_stamp: "Formatted at 2026-08-12 03:56:42 on dist-test-slave-tm1g"
server_key: "009f300348a0b04041e2bab637b34f1d"
server_key_iv: "5a03da1aca609e921007d0bcc3217ac8"
server_key_version: "encryptionkey@0"
I20260812 03:56:42.832332 8851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:42.839382 8851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:42.841507 8851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:42.841691 8851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:42.846037 8851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43371
I20260812 03:56:42.846073 8930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43371 every 8 connection(s)
I20260812 03:56:42.846517 8851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:42.847435 8931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:42.847676 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 8851
I20260812 03:56:42.847813 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:42.848011 17627 external_mini_cluster.cc:1428] Setting key 2ab51a29628a9a6a6bc8909c1d996537
I20260812 03:56:42.851239 8931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395: Bootstrap starting.
I20260812 03:56:42.851905 8931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:42.852239 8931 log.cc:824] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:42.853104 8931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395: No bootstrap required, opened a new log
I20260812 03:56:42.854378 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:42.848584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35565 (local address 127.0.0.1:43371)
0812 03:56:42.848802 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:42.848812 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:42.848849 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:56:42.849347 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:42.849357 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:42.849421 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:42.849587 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:42.849817 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:42.851222 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:42.852177 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:42.852751 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:42.852865 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:42.853593 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:42.853597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:42.853598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:42.853812 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:42.853853 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:42.853869 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:42.853947 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:42.854045 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 03:56:42.854115 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":103,"threads_started":1}
I20260812 03:56:42.855479 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:42.855736 8931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0837fd65fb4bd3a1a3f6e827d27395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } }
I20260812 03:56:42.855893 8931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:42.855940 8931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0837fd65fb4bd3a1a3f6e827d27395, State: Initialized, Role: FOLLOWER
I20260812 03:56:42.856074 8931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0837fd65fb4bd3a1a3f6e827d27395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } }
I20260812 03:56:42.856173 8931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:42.856217 8931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:42.856267 8931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:42.857012 8931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0837fd65fb4bd3a1a3f6e827d27395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } }
I20260812 03:56:42.857127 8931 leader_election.cc:302] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b0837fd65fb4bd3a1a3f6e827d27395; no voters:
I20260812 03:56:42.857291 8931 leader_election.cc:288] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:42.857323 8941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:42.857434 8941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [term 1 LEADER]: Becoming Leader. State: Replica: 4b0837fd65fb4bd3a1a3f6e827d27395, State: Running, Role: LEADER
I20260812 03:56:42.857558 8931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:42.857577 8941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0837fd65fb4bd3a1a3f6e827d27395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } }
I20260812 03:56:42.857955 8944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b0837fd65fb4bd3a1a3f6e827d27395. Latest consensus state: current_term: 1 leader_uuid: "4b0837fd65fb4bd3a1a3f6e827d27395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0837fd65fb4bd3a1a3f6e827d27395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } } }
I20260812 03:56:42.857970 8943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b0837fd65fb4bd3a1a3f6e827d27395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0837fd65fb4bd3a1a3f6e827d27395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } } }
I20260812 03:56:42.858083 8944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:42.858397 8943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395 [sys.catalog]: This master's current role is: LEADER
W20260812 03:56:42.859632 8960 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:56:42.859690 8960 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:56:42.859735 8949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:42.860244 8949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:42.860486 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:42.857038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49493 (local address 127.0.0.1:43371)
0812 03:56:42.857082 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:42.857084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:42.857125 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:56:42.857198 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:42.857199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:42.857210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:42.857255 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:42.857410 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:42.858077 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:42.859128 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:42.859442 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:42.859528 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:42.860235 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:42.860237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:42.860237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:42.860242 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:42.860256 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:42.860260 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:42.860309 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:42.860374 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:56:42.860399 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:56:42.861765 8949 catalog_manager.cc:1347] Generated new cluster ID: e87f3023a8a74d4390b4babe264be66b
I20260812 03:56:42.861826 8949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:42.878513 8949 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:42.879112 8949 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:42.884092 8949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b0837fd65fb4bd3a1a3f6e827d27395: Generated new TSK 0
I20260812 03:56:42.884303 8949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:42.893028 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 8851
2026-08-12T03:56:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:42 dist-test-slave-tm1g krb5kdc[8966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:42 dist-test-slave-tm1g krb5kdc[8966](info): set up 2 sockets
Aug 12 03:56:42 dist-test-slave-tm1g krb5kdc[8966](info): commencing operation
krb5kdc: starting...
W20260812 03:56:44.014570 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:44 dist-test-slave-tm1g krb5kdc[8966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:44.146567 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:44.229954 8979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:44.230157 8979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:44.230181 8979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:44.231398 8979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:44.231444 8979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:44.231458 8979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:44.231509 8979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:44.231534 8979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:44.233067 8979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:44.233340 8979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:44.233589 8979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:44.238734 8987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:44.238734 8988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:44.238768 8990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:44.239032 8979 server_base.cc:1034] running on GCE node
I20260812 03:56:44.239326 8979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:44.239636 8979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:44.240804 8979 hybrid_clock.cc:648] HybridClock initialized: now 1786507004240771 us; error 54 us; skew 500 ppm
Aug 12 03:56:44 dist-test-slave-tm1g krb5kdc[8966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507004, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:44.243511 8979 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:44.243960 8979 webserver.cc:460] Webserver started at http://127.0.0.1:38739/ using document root <none> and password file <none>
I20260812 03:56:44.244217 8979 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:44.244285 8979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:44.244433 8979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:44.245604 8979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "3fcba2fe229e4f1982d60dd54ff440ab"
format_stamp: "Formatted at 2026-08-12 03:56:44 on dist-test-slave-tm1g"
server_key: "da1fca7b3a19d40670ad79e24ec0cd63"
server_key_iv: "658156f135102ca3de02a08c2c9ed975"
server_key_version: "encryptionkey@0"
I20260812 03:56:44.245960 8979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "3fcba2fe229e4f1982d60dd54ff440ab"
format_stamp: "Formatted at 2026-08-12 03:56:44 on dist-test-slave-tm1g"
server_key: "da1fca7b3a19d40670ad79e24ec0cd63"
server_key_iv: "658156f135102ca3de02a08c2c9ed975"
server_key_version: "encryptionkey@0"
I20260812 03:56:44.247413 8979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:44.248216 9000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:44.248466 8979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:44.248586 8979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "3fcba2fe229e4f1982d60dd54ff440ab"
format_stamp: "Formatted at 2026-08-12 03:56:44 on dist-test-slave-tm1g"
server_key: "da1fca7b3a19d40670ad79e24ec0cd63"
server_key_iv: "658156f135102ca3de02a08c2c9ed975"
server_key_version: "encryptionkey@0"
I20260812 03:56:44.248677 8979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:44.252170 8979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:44.254151 8979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:44.254401 8979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:44.259096 8979 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33797
I20260812 03:56:44.259207 9060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33797 every 8 connection(s)
I20260812 03:56:44.259526 8979 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:44.260692 9061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:44.262162 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 8979
I20260812 03:56:44.262257 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:44.262485 17627 external_mini_cluster.cc:1428] Setting key f035e0511033fe2c5a8753c864eae749
I20260812 03:56:44.263835 9061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab: Bootstrap starting.
I20260812 03:56:44.264643 9061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:44.265058 9061 log.cc:824] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:44.265337 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:44.263040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49929 (local address 127.0.0.1:33797)
0812 03:56:44.263343 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:44.263353 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:44.263391 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:56:44.264010 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:44.264019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:44.264054 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:44.264232 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:44.264867 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:44.264872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:44.264872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:44.264877 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:44.264930 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:44.264946 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:44.265038 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:44.265073 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 03:56:44.265117 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":69,"threads_started":1}
I20260812 03:56:44.266043 9061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab: No bootstrap required, opened a new log
I20260812 03:56:44.266464 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:44.268708 9061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcba2fe229e4f1982d60dd54ff440ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260812 03:56:44.268926 9061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:44.269029 9061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcba2fe229e4f1982d60dd54ff440ab, State: Initialized, Role: FOLLOWER
I20260812 03:56:44.269104 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:44.267901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47567 (local address 127.0.0.1:33797)
0812 03:56:44.267952 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:44.267955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:44.268011 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:56:44.268072 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:44.268073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:44.268079 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:44.268165 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:44.268732 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:44.268734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:44.268734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:44.268739 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:44.268754 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:44.268759 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:44.268821 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:44.268982 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:56:44.269001 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:56:44.269236 9061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcba2fe229e4f1982d60dd54ff440ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260812 03:56:44.269366 9061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:44.269420 9061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:44.269477 9061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:44.270085 9061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcba2fe229e4f1982d60dd54ff440ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260812 03:56:44.270215 9061 leader_election.cc:302] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fcba2fe229e4f1982d60dd54ff440ab; no voters:
I20260812 03:56:44.270416 9061 leader_election.cc:288] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:44.270439 9072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:44.270613 9072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [term 1 LEADER]: Becoming Leader. State: Replica: 3fcba2fe229e4f1982d60dd54ff440ab, State: Running, Role: LEADER
I20260812 03:56:44.270694 9061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:44.270753 9072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcba2fe229e4f1982d60dd54ff440ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260812 03:56:44.271134 9074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fcba2fe229e4f1982d60dd54ff440ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcba2fe229e4f1982d60dd54ff440ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } } }
I20260812 03:56:44.271252 9074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:44.271133 9075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fcba2fe229e4f1982d60dd54ff440ab. Latest consensus state: current_term: 1 leader_uuid: "3fcba2fe229e4f1982d60dd54ff440ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcba2fe229e4f1982d60dd54ff440ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } } }
I20260812 03:56:44.271421 9075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:44.271947 9080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:44.272466 9080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:44.274080 9080 catalog_manager.cc:1347] Generated new cluster ID: 64bdb44e764d48b8a205e4dcd5e47770
I20260812 03:56:44.274137 9080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:44.289597 9080 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:44.289783 9080 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:44.295306 9080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fcba2fe229e4f1982d60dd54ff440ab: Generated new TSK 0
I20260812 03:56:44.295499 9080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:44.303027 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 8979
2026-08-12T03:56:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:44 dist-test-slave-tm1g krb5kdc[9098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:44 dist-test-slave-tm1g krb5kdc[9098](info): set up 2 sockets
Aug 12 03:56:44 dist-test-slave-tm1g krb5kdc[9098](info): commencing operation
krb5kdc: starting...
W20260812 03:56:45.425863 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:45 dist-test-slave-tm1g krb5kdc[9098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:45.553550 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:45.635659 9115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:45.635893 9115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:45.635931 9115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:45.637091 9115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:45.637164 9115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:45.637200 9115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:45.637261 9115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:45.637285 9115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:45.638854 9115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:45.639129 9115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:45.639364 9115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:45.644588 9120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:45.644577 9123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:45.644583 9121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:45.644618 9115 server_base.cc:1034] running on GCE node
I20260812 03:56:45.645124 9115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:45.645467 9115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:45.646665 9115 hybrid_clock.cc:648] HybridClock initialized: now 1786507005646661 us; error 59 us; skew 500 ppm
Aug 12 03:56:45 dist-test-slave-tm1g krb5kdc[9098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:45.649348 9115 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:45.649822 9115 webserver.cc:460] Webserver started at http://127.0.0.1:43133/ using document root <none> and password file <none>
I20260812 03:56:45.650084 9115 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:45.650153 9115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:45.650339 9115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:45.651432 9115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "e40999810b9d4eb399b3804ce4f84c0a"
format_stamp: "Formatted at 2026-08-12 03:56:45 on dist-test-slave-tm1g"
server_key: "032b449b87bbf935a1d53cb79d961b6f"
server_key_iv: "023b3ce144c1f15f870280db53671169"
server_key_version: "encryptionkey@0"
I20260812 03:56:45.651793 9115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "e40999810b9d4eb399b3804ce4f84c0a"
format_stamp: "Formatted at 2026-08-12 03:56:45 on dist-test-slave-tm1g"
server_key: "032b449b87bbf935a1d53cb79d961b6f"
server_key_iv: "023b3ce144c1f15f870280db53671169"
server_key_version: "encryptionkey@0"
I20260812 03:56:45.653142 9115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:45.653868 9130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:45.654042 9115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:56:45.654151 9115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "e40999810b9d4eb399b3804ce4f84c0a"
format_stamp: "Formatted at 2026-08-12 03:56:45 on dist-test-slave-tm1g"
server_key: "032b449b87bbf935a1d53cb79d961b6f"
server_key_iv: "023b3ce144c1f15f870280db53671169"
server_key_version: "encryptionkey@0"
I20260812 03:56:45.654227 9115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:45.670197 9115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:45.672525 9115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:45.672705 9115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:45.677327 9115 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44815
I20260812 03:56:45.677383 9189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44815 every 8 connection(s)
I20260812 03:56:45.677783 9115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:45.678871 9191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:45.679179 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 9115
I20260812 03:56:45.679307 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:45.679505 17627 external_mini_cluster.cc:1428] Setting key 29016eb1ad91d31f8bff169db7bc3145
I20260812 03:56:45.682690 9191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a: Bootstrap starting.
I20260812 03:56:45.683511 9191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:45.683844 9191 log.cc:824] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:45.684687 9191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a: No bootstrap required, opened a new log
I20260812 03:56:45.686857 9191 raft_consensus.cc:348] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40999810b9d4eb399b3804ce4f84c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44815 } }
I20260812 03:56:45.687032 9191 raft_consensus.cc:374] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 12 03:56:45 dist-test-slave-tm1g krb5kdc[9098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:56:45.687086 9191 raft_consensus.cc:729] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40999810b9d4eb399b3804ce4f84c0a, State: Initialized, Role: FOLLOWER
I20260812 03:56:45.687235 9191 consensus_queue.cc:260] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40999810b9d4eb399b3804ce4f84c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44815 } }
I20260812 03:56:45.687342 9191 raft_consensus.cc:388] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:45.687386 9191 raft_consensus.cc:482] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:45.687450 9191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:45.688103 9191 raft_consensus.cc:504] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40999810b9d4eb399b3804ce4f84c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44815 } }
I20260812 03:56:45.688239 9191 leader_election.cc:302] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e40999810b9d4eb399b3804ce4f84c0a; no voters:
I20260812 03:56:45.688452 9191 leader_election.cc:288] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:45.688506 9199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:45.688674 9199 raft_consensus.cc:686] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [term 1 LEADER]: Becoming Leader. State: Replica: e40999810b9d4eb399b3804ce4f84c0a, State: Running, Role: LEADER
I20260812 03:56:45.688751 9191 sys_catalog.cc:564] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:45.688789 9199 consensus_queue.cc:237] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40999810b9d4eb399b3804ce4f84c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44815 } }
I20260812 03:56:45.689189 9199 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e40999810b9d4eb399b3804ce4f84c0a. Latest consensus state: current_term: 1 leader_uuid: "e40999810b9d4eb399b3804ce4f84c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40999810b9d4eb399b3804ce4f84c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44815 } } }
I20260812 03:56:45.689349 9199 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:45.689849 9210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:45.690488 9210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:45.690846 9201 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e40999810b9d4eb399b3804ce4f84c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40999810b9d4eb399b3804ce4f84c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44815 } } }
I20260812 03:56:45.691280 9201 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:45.691877 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:45.680091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:56217 (local address 127.0.0.1:44815)
0812 03:56:45.680261 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:45.680268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:45.680296 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:56:45.680740 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:45.680747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:45.680797 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:45.680910 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:45.681820 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:45.682897 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:45.684079 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:45.684678 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:45.684839 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:45.687742 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:45.687746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:45.687746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:45.687757 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:45.689750 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:45.690251 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:45.690254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:45.690255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:45.690387 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:45.690649 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:45.690650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:45.690650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:45.690934 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:45.691083 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:45.691574 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 03:56:45.691675 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260812 03:56:45.692567 9210 catalog_manager.cc:1347] Generated new cluster ID: 21175609133344f291f55d26abbde28f
I20260812 03:56:45.692624 9210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:45.692960 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:45.702517 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:45.694597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:35185 (local address 127.0.0.1:44815)
0812 03:56:45.694632 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:45.694634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:45.694647 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:56:45.694729 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:45.694730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:45.694740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:45.694791 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:45.695430 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:45.696502 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:45.697587 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:45.697921 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:45.697998 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:45.699441 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:45.699442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:45.699442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:45.699450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:45.701061 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:45.701501 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:45.701502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:45.701502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:45.701562 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:45.701799 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:45.701800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:45.701800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:45.701936 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:45.702043 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:45.702188 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:56:45.702255 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:56:45.704800 9210 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:45.705386 9210 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:45.716378 9210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e40999810b9d4eb399b3804ce4f84c0a: Generated new TSK 0
I20260812 03:56:45.716626 9210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:45.718757 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 9115
W20260812 03:56:45.728904 9095 connection.cc:535] client connection to 127.0.0.1:44815 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44815): Connection reset by peer (error 104)
2026-08-12T03:56:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:45 dist-test-slave-tm1g krb5kdc[9227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:45 dist-test-slave-tm1g krb5kdc[9227](info): set up 2 sockets
Aug 12 03:56:45 dist-test-slave-tm1g krb5kdc[9227](info): commencing operation
krb5kdc: starting...
W20260812 03:56:46.853564 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:46 dist-test-slave-tm1g krb5kdc[9227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:46.985692 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42167
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:47.068773 9245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:47.068985 9245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:47.069010 9245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:47.070202 9245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:47.070247 9245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:47.070261 9245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:47.070274 9245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:47.070287 9245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:47.071897 9245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:47.072149 9245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:47.072407 9245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:47.077700 9250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:47.077685 9254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:47.077672 9251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:47.077767 9245 server_base.cc:1034] running on GCE node
I20260812 03:56:47.078146 9245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:47.078502 9245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:47.079707 9245 hybrid_clock.cc:648] HybridClock initialized: now 1786507007079693 us; error 30 us; skew 500 ppm
Aug 12 03:56:47 dist-test-slave-tm1g krb5kdc[9227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507007, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:47.082592 9245 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:47.083079 9245 webserver.cc:460] Webserver started at http://127.0.0.1:32869/ using document root <none> and password file <none>
I20260812 03:56:47.083371 9245 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:47.083451 9245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:47.083617 9245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:47.084813 9245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "80b28a14539b49ce931331c26c0e0859"
format_stamp: "Formatted at 2026-08-12 03:56:47 on dist-test-slave-tm1g"
server_key: "83a508e781e241823283e6dee155c977"
server_key_iv: "62ea4a3063b37d5d02f374335f3db046"
server_key_version: "encryptionkey@0"
I20260812 03:56:47.085209 9245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "80b28a14539b49ce931331c26c0e0859"
format_stamp: "Formatted at 2026-08-12 03:56:47 on dist-test-slave-tm1g"
server_key: "83a508e781e241823283e6dee155c977"
server_key_iv: "62ea4a3063b37d5d02f374335f3db046"
server_key_version: "encryptionkey@0"
I20260812 03:56:47.086740 9245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:47.087636 9261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:47.087870 9245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:47.087996 9245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "80b28a14539b49ce931331c26c0e0859"
format_stamp: "Formatted at 2026-08-12 03:56:47 on dist-test-slave-tm1g"
server_key: "83a508e781e241823283e6dee155c977"
server_key_iv: "62ea4a3063b37d5d02f374335f3db046"
server_key_version: "encryptionkey@0"
I20260812 03:56:47.088080 9245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:47.104202 9245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:47.106081 9245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:47.106249 9245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:47.110246 9245 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42167
I20260812 03:56:47.110387 9320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42167 every 8 connection(s)
I20260812 03:56:47.110726 9245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:47.111202 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 9245
I20260812 03:56:47.111342 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:47.111558 17627 external_mini_cluster.cc:1428] Setting key a98f22cdabc86ba818a9ccf4cb7fe35d
I20260812 03:56:47.111780 9322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:47.114827 9322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859: Bootstrap starting.
I20260812 03:56:47.115490 9322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:47.115804 9322 log.cc:824] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:47.116549 9322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859: No bootstrap required, opened a new log
I20260812 03:56:47.117201 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:47.112137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:43003 (local address 127.0.0.1:42167)
0812 03:56:47.112339 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:47.112347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:47.112371 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:56:47.112930 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:47.112937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:47.112987 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:47.113135 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:47.113379 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:47.114286 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:47.115180 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:47.115622 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:47.115745 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:47.116531 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:47.116537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:47.116538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:47.116754 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:47.116786 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:47.116798 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:47.116898 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:47.116982 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:56:47.117013 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260812 03:56:47.118266 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:47.118872 9322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b28a14539b49ce931331c26c0e0859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42167 } }
I20260812 03:56:47.119032 9322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:47.119079 9322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b28a14539b49ce931331c26c0e0859, State: Initialized, Role: FOLLOWER
I20260812 03:56:47.119206 9322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b28a14539b49ce931331c26c0e0859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42167 } }
I20260812 03:56:47.119295 9322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:47.119334 9322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:47.119391 9322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:47.120209 9322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b28a14539b49ce931331c26c0e0859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42167 } }
I20260812 03:56:47.120324 9322 leader_election.cc:302] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80b28a14539b49ce931331c26c0e0859; no voters:
I20260812 03:56:47.120486 9322 leader_election.cc:288] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:47.120554 9334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:47.120724 9322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:47.120743 9334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [term 1 LEADER]: Becoming Leader. State: Replica: 80b28a14539b49ce931331c26c0e0859, State: Running, Role: LEADER
I20260812 03:56:47.120829 9334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b28a14539b49ce931331c26c0e0859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42167 } }
I20260812 03:56:47.121155 9336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80b28a14539b49ce931331c26c0e0859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b28a14539b49ce931331c26c0e0859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42167 } } }
I20260812 03:56:47.121245 9336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:47.121142 9334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80b28a14539b49ce931331c26c0e0859. Latest consensus state: current_term: 1 leader_uuid: "80b28a14539b49ce931331c26c0e0859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b28a14539b49ce931331c26c0e0859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42167 } } }
I20260812 03:56:47.121477 9334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:47.121596 9340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:47.122430 9340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:47.123724 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:47.119777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:54775 (local address 127.0.0.1:42167)
0812 03:56:47.119824 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:47.119826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:47.119916 (+ 90us) server_negotiation.cc:407] Connection header received
0812 03:56:47.119959 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:47.119960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:47.119970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:47.120024 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:47.120144 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:47.120836 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:47.122012 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:47.122482 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:47.122554 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:47.123419 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:47.123420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:47.123421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:47.123425 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:47.123443 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:47.123448 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:47.123505 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:47.123615 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 03:56:47.123628 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:56:47.124202 9340 catalog_manager.cc:1347] Generated new cluster ID: c302b3d5ce7146c88effc8ee3ad2ca7a
I20260812 03:56:47.124253 9340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:47.139907 9340 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:47.140501 9340 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:47.152386 9340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80b28a14539b49ce931331c26c0e0859: Generated new TSK 0
I20260812 03:56:47.152618 9340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:47.162006 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 9245
W20260812 03:56:47.171031 9225 connection.cc:535] client connection to 127.0.0.1:42167 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42167): Connection reset by peer (error 104)
2026-08-12T03:56:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:47 dist-test-slave-tm1g krb5kdc[9358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:47 dist-test-slave-tm1g krb5kdc[9358](info): set up 2 sockets
Aug 12 03:56:47 dist-test-slave-tm1g krb5kdc[9358](info): commencing operation
krb5kdc: starting...
W20260812 03:56:48.280004 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:48 dist-test-slave-tm1g krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:48.413833 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:48.499203 9375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:48.499425 9375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:48.499450 9375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:48.500581 9375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:48.500618 9375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:48.500633 9375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:48.500645 9375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:48.500658 9375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:48.502115 9375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:48.502405 9375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:48.502635 9375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:48.507719 9381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:48.507718 9383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:48.507777 9380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:48.508044 9375 server_base.cc:1034] running on GCE node
I20260812 03:56:48.508359 9375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:48.508692 9375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:48.509876 9375 hybrid_clock.cc:648] HybridClock initialized: now 1786507008509849 us; error 36 us; skew 500 ppm
Aug 12 03:56:48 dist-test-slave-tm1g krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507008, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:48.520259 9375 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:48.520819 9375 webserver.cc:460] Webserver started at http://127.0.0.1:46671/ using document root <none> and password file <none>
I20260812 03:56:48.521098 9375 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:48.521171 9375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:48.521324 9375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:48.522553 9375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "424642d1ab5e445087bc5e2f19919707"
format_stamp: "Formatted at 2026-08-12 03:56:48 on dist-test-slave-tm1g"
server_key: "5a601ad08c019f17841fb52c57d0b8f1"
server_key_iv: "235142083ea2069720208d0202e08e15"
server_key_version: "encryptionkey@0"
I20260812 03:56:48.522940 9375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "424642d1ab5e445087bc5e2f19919707"
format_stamp: "Formatted at 2026-08-12 03:56:48 on dist-test-slave-tm1g"
server_key: "5a601ad08c019f17841fb52c57d0b8f1"
server_key_iv: "235142083ea2069720208d0202e08e15"
server_key_version: "encryptionkey@0"
I20260812 03:56:48.524389 9375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:56:48.525163 9395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:48.525375 9375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:48.525490 9375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "424642d1ab5e445087bc5e2f19919707"
format_stamp: "Formatted at 2026-08-12 03:56:48 on dist-test-slave-tm1g"
server_key: "5a601ad08c019f17841fb52c57d0b8f1"
server_key_iv: "235142083ea2069720208d0202e08e15"
server_key_version: "encryptionkey@0"
I20260812 03:56:48.525565 9375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:48.529086 9375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:48.531477 9375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:48.531651 9375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:48.536140 9375 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46487
I20260812 03:56:48.536172 9454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46487 every 8 connection(s)
I20260812 03:56:48.536502 9375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:48.537424 9455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:48.539764 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 9375
I20260812 03:56:48.539795 9455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707: Bootstrap starting.
I20260812 03:56:48.539896 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:48.540097 17627 external_mini_cluster.cc:1428] Setting key 704a30faa62bb53dae359f067dfa92db
I20260812 03:56:48.540556 9455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:48.540946 9455 log.cc:824] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:48.541816 9455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707: No bootstrap required, opened a new log
I20260812 03:56:48.543184 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:48.540692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:53897 (local address 127.0.0.1:46487)
0812 03:56:48.540925 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:48.540933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:48.540967 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:56:48.541650 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:48.541658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:48.541687 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:48.541846 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:48.542691 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:48.542694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:48.542694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:48.542698 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:48.542742 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:48.542753 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:48.542843 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:48.542915 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:56:48.542985 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":116,"threads_started":1}
I20260812 03:56:48.544432 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:48.544488 9455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424642d1ab5e445087bc5e2f19919707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46487 } }
I20260812 03:56:48.544667 9455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:48.544711 9455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 424642d1ab5e445087bc5e2f19919707, State: Initialized, Role: FOLLOWER
I20260812 03:56:48.544831 9455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424642d1ab5e445087bc5e2f19919707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46487 } }
I20260812 03:56:48.544910 9455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:48.544932 9455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:48.544999 9455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:48.545631 9455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424642d1ab5e445087bc5e2f19919707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46487 } }
I20260812 03:56:48.545764 9455 leader_election.cc:302] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 424642d1ab5e445087bc5e2f19919707; no voters:
I20260812 03:56:48.545938 9455 leader_election.cc:288] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:48.546041 9465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:48.546190 9465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [term 1 LEADER]: Becoming Leader. State: Replica: 424642d1ab5e445087bc5e2f19919707, State: Running, Role: LEADER
I20260812 03:56:48.546202 9455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:48.546366 9465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424642d1ab5e445087bc5e2f19919707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46487 } }
I20260812 03:56:48.546723 9468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "424642d1ab5e445087bc5e2f19919707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424642d1ab5e445087bc5e2f19919707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46487 } } }
I20260812 03:56:48.546886 9468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:48.546749 9467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 424642d1ab5e445087bc5e2f19919707. Latest consensus state: current_term: 1 leader_uuid: "424642d1ab5e445087bc5e2f19919707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424642d1ab5e445087bc5e2f19919707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46487 } } }
I20260812 03:56:48.547190 9467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:48.547511 9479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:48.547667 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:48.546030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:43437 (local address 127.0.0.1:46487)
0812 03:56:48.546067 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:48.546069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:48.546431 (+ 362us) server_negotiation.cc:407] Connection header received
0812 03:56:48.546454 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:48.546455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:48.546461 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:48.546504 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:48.547296 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:48.547297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:48.547298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:48.547302 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:48.547327 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:48.547332 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:48.547386 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:48.547562 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 03:56:48.547578 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:56:48.548182 9479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:48.549883 9479 catalog_manager.cc:1347] Generated new cluster ID: 77b159a654494aac8003a8f1470c9202
I20260812 03:56:48.549942 9479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:48.562006 9479 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:48.562187 9479 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:48.581992 9479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 424642d1ab5e445087bc5e2f19919707: Generated new TSK 0
I20260812 03:56:48.582223 9479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:48.584975 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 9375
2026-08-12T03:56:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:48 dist-test-slave-tm1g krb5kdc[9493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:48 dist-test-slave-tm1g krb5kdc[9493](info): set up 2 sockets
Aug 12 03:56:48 dist-test-slave-tm1g krb5kdc[9493](info): commencing operation
krb5kdc: starting...
W20260812 03:56:49.698112 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:49 dist-test-slave-tm1g krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:49.829867 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:49.920347 9507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:49.920586 9507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:49.920614 9507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:49.921916 9507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:49.921969 9507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:49.921984 9507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:49.921998 9507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:49.922010 9507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:49.923739 9507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44305
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:49.924134 9507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:49.924407 9507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:49.930083 9515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:49.930083 9512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:49.930078 9513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:49.930505 9507 server_base.cc:1034] running on GCE node
I20260812 03:56:49.930787 9507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:49.931103 9507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:49.932289 9507 hybrid_clock.cc:648] HybridClock initialized: now 1786507009932294 us; error 39 us; skew 500 ppm
Aug 12 03:56:49 dist-test-slave-tm1g krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:49.935128 9507 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:49.935695 9507 webserver.cc:460] Webserver started at http://127.0.0.1:35695/ using document root <none> and password file <none>
I20260812 03:56:49.935942 9507 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:49.935987 9507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:49.936151 9507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:49.937282 9507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "ece2624573714cc788e371c1b18f3720"
format_stamp: "Formatted at 2026-08-12 03:56:49 on dist-test-slave-tm1g"
server_key: "1d8ea90756296473fc141df5c5d19506"
server_key_iv: "e85162be09abfcf633052c7c658230ea"
server_key_version: "encryptionkey@0"
I20260812 03:56:49.937661 9507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "ece2624573714cc788e371c1b18f3720"
format_stamp: "Formatted at 2026-08-12 03:56:49 on dist-test-slave-tm1g"
server_key: "1d8ea90756296473fc141df5c5d19506"
server_key_iv: "e85162be09abfcf633052c7c658230ea"
server_key_version: "encryptionkey@0"
I20260812 03:56:49.939193 9507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:56:49.940140 9524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:49.940357 9507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:49.940483 9507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "ece2624573714cc788e371c1b18f3720"
format_stamp: "Formatted at 2026-08-12 03:56:49 on dist-test-slave-tm1g"
server_key: "1d8ea90756296473fc141df5c5d19506"
server_key_iv: "e85162be09abfcf633052c7c658230ea"
server_key_version: "encryptionkey@0"
I20260812 03:56:49.940599 9507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:49.975533 9507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:49.977890 9507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:49.978084 9507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:49.982220 9507 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44305
I20260812 03:56:49.982261 9581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44305 every 8 connection(s)
I20260812 03:56:49.982724 9507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:49.983626 9582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:49.985932 9582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720: Bootstrap starting.
I20260812 03:56:49.986454 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 9507
I20260812 03:56:49.986531 9582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:49.986577 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:49.986758 17627 external_mini_cluster.cc:1428] Setting key 37a4832d7c034e59d63e37dfeffbbf2c
I20260812 03:56:49.986884 9582 log.cc:824] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:49.987783 9582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720: No bootstrap required, opened a new log
I20260812 03:56:49.990355 9582 raft_consensus.cc:348] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece2624573714cc788e371c1b18f3720" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44305 } }
I20260812 03:56:49.990526 9582 raft_consensus.cc:374] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:49.990561 9582 raft_consensus.cc:729] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece2624573714cc788e371c1b18f3720, State: Initialized, Role: FOLLOWER
I20260812 03:56:49.990691 9582 consensus_queue.cc:260] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece2624573714cc788e371c1b18f3720" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44305 } }
I20260812 03:56:49.990779 9582 raft_consensus.cc:388] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:49.990803 9582 raft_consensus.cc:482] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:49.990875 9582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:49.991485 9582 raft_consensus.cc:504] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece2624573714cc788e371c1b18f3720" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44305 } }
I20260812 03:56:49.991590 9582 leader_election.cc:302] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ece2624573714cc788e371c1b18f3720; no voters:
I20260812 03:56:49.991750 9582 leader_election.cc:288] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:49.991799 9587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:49.991954 9587 raft_consensus.cc:686] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [term 1 LEADER]: Becoming Leader. State: Replica: ece2624573714cc788e371c1b18f3720, State: Running, Role: LEADER
I20260812 03:56:49.992003 9582 sys_catalog.cc:564] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:49.992041 9587 consensus_queue.cc:237] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece2624573714cc788e371c1b18f3720" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44305 } }
I20260812 03:56:49.992362 9587 sys_catalog.cc:455] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ece2624573714cc788e371c1b18f3720. Latest consensus state: current_term: 1 leader_uuid: "ece2624573714cc788e371c1b18f3720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece2624573714cc788e371c1b18f3720" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44305 } } }
I20260812 03:56:49.992453 9587 sys_catalog.cc:458] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:49.992370 9588 sys_catalog.cc:455] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ece2624573714cc788e371c1b18f3720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece2624573714cc788e371c1b18f3720" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44305 } } }
I20260812 03:56:49.992623 9588 sys_catalog.cc:458] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:49.993330 9593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:49.993866 9593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:56:49 dist-test-slave-tm1g krb5kdc[9493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:56:49.995575 9593 catalog_manager.cc:1347] Generated new cluster ID: 161c14551d1d46929b6f70a9923b3e6a
I20260812 03:56:49.995622 9593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:49.998708 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:49.987344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:44517 (local address 127.0.0.1:44305)
0812 03:56:49.987494 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:49.987504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:49.987538 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:56:49.988195 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:49.988204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:49.988270 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:49.988423 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:49.989072 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:49.990580 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:49.991419 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:49.992066 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:49.994123 (+ 2057us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:49.995364 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:49.995368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:49.995368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:49.995380 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:49.997119 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:49.997529 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:49.997535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:49.997535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:49.997625 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:49.997863 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:49.997864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:49.997864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:49.998100 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:49.998300 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:49.998426 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 03:56:49.998546 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260812 03:56:50.003647 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:50.008275 9593 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:50.008975 9593 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:50.012826 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:50.006050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:58659 (local address 127.0.0.1:44305)
0812 03:56:50.006094 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:50.006096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:50.006258 (+ 162us) server_negotiation.cc:407] Connection header received
0812 03:56:50.006362 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:50.006363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:50.006374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:50.006425 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:50.007062 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:50.007584 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:50.008305 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:50.008581 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:50.008664 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:50.009967 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:50.009969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:50.009969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:50.009977 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:50.011534 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:50.011926 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:50.011927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:50.011928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:50.011999 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:50.012242 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:50.012243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:50.012244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:50.012362 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:50.012495 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:50.012621 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 03:56:50.012691 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:56:50.016271 9593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ece2624573714cc788e371c1b18f3720: Generated new TSK 0
I20260812 03:56:50.016474 9593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:50.018782 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 9507
W20260812 03:56:50.025672 9491 connection.cc:535] client connection to 127.0.0.1:44305 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44305): Connection reset by peer (error 104)
2026-08-12T03:56:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:50 dist-test-slave-tm1g krb5kdc[9619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:50 dist-test-slave-tm1g krb5kdc[9619](info): set up 2 sockets
Aug 12 03:56:50 dist-test-slave-tm1g krb5kdc[9619](info): commencing operation
krb5kdc: starting...
W20260812 03:56:51.153473 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:51 dist-test-slave-tm1g krb5kdc[9619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:51.282207 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:51.365360 9634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:51.365579 9634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:51.365604 9634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:51.366837 9634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:51.366880 9634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:51.366895 9634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:51.366907 9634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:51.366920 9634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:51.368466 9634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35141
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:51.368728 9634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:51.368979 9634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:51.374155 9643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:51.374173 9641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:51.374152 9640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:51.374353 9634 server_base.cc:1034] running on GCE node
I20260812 03:56:51.374684 9634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:51.374989 9634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:51.376165 9634 hybrid_clock.cc:648] HybridClock initialized: now 1786507011376149 us; error 30 us; skew 500 ppm
Aug 12 03:56:51 dist-test-slave-tm1g krb5kdc[9619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:51.378875 9634 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:51.379320 9634 webserver.cc:460] Webserver started at http://127.0.0.1:42683/ using document root <none> and password file <none>
I20260812 03:56:51.379513 9634 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:51.379554 9634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:51.379642 9634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:51.380671 9634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "2d57a6e29f6b4394a337b30e26ea2dc5"
format_stamp: "Formatted at 2026-08-12 03:56:51 on dist-test-slave-tm1g"
server_key: "05d50e0b9a27b7fcf29bc1838a5c13a4"
server_key_iv: "8c1ee2a17cf1fff2ea5818d68ff5282f"
server_key_version: "encryptionkey@0"
I20260812 03:56:51.380980 9634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "2d57a6e29f6b4394a337b30e26ea2dc5"
format_stamp: "Formatted at 2026-08-12 03:56:51 on dist-test-slave-tm1g"
server_key: "05d50e0b9a27b7fcf29bc1838a5c13a4"
server_key_iv: "8c1ee2a17cf1fff2ea5818d68ff5282f"
server_key_version: "encryptionkey@0"
I20260812 03:56:51.382261 9634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:56:51.383080 9652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:51.383237 9634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:56:51.383328 9634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "2d57a6e29f6b4394a337b30e26ea2dc5"
format_stamp: "Formatted at 2026-08-12 03:56:51 on dist-test-slave-tm1g"
server_key: "05d50e0b9a27b7fcf29bc1838a5c13a4"
server_key_iv: "8c1ee2a17cf1fff2ea5818d68ff5282f"
server_key_version: "encryptionkey@0"
I20260812 03:56:51.383386 9634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:51.408352 9634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:51.410585 9634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:51.410704 9634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:51.414767 9634 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35141
I20260812 03:56:51.414866 9713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35141 every 8 connection(s)
I20260812 03:56:51.415220 9634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:51.416167 9714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:51.418531 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 9634
I20260812 03:56:51.418663 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:51.418680 9714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5: Bootstrap starting.
I20260812 03:56:51.418859 17627 external_mini_cluster.cc:1428] Setting key 2fff2421b00d9dd6d8b1eba9a076398e
I20260812 03:56:51.419492 9714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:51.419871 9714 log.cc:824] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:51.420699 9714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5: No bootstrap required, opened a new log
I20260812 03:56:51.422712 9714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d57a6e29f6b4394a337b30e26ea2dc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35141 } }
I20260812 03:56:51.422879 9714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:51.422927 9714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d57a6e29f6b4394a337b30e26ea2dc5, State: Initialized, Role: FOLLOWER
I20260812 03:56:51.423094 9714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d57a6e29f6b4394a337b30e26ea2dc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35141 } }
I20260812 03:56:51.423194 9714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:51.423239 9714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:51.423288 9714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:51.423995 9714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d57a6e29f6b4394a337b30e26ea2dc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35141 } }
I20260812 03:56:51.424113 9714 leader_election.cc:302] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d57a6e29f6b4394a337b30e26ea2dc5; no voters:
I20260812 03:56:51.424309 9714 leader_election.cc:288] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:51.424335 9719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:51.424472 9719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [term 1 LEADER]: Becoming Leader. State: Replica: 2d57a6e29f6b4394a337b30e26ea2dc5, State: Running, Role: LEADER
I20260812 03:56:51.424595 9714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:51.424614 9719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d57a6e29f6b4394a337b30e26ea2dc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35141 } }
I20260812 03:56:51.425053 9719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d57a6e29f6b4394a337b30e26ea2dc5. Latest consensus state: current_term: 1 leader_uuid: "2d57a6e29f6b4394a337b30e26ea2dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d57a6e29f6b4394a337b30e26ea2dc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35141 } } }
I20260812 03:56:51.425130 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:51.419442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:44117 (local address 127.0.0.1:35141)
0812 03:56:51.419726 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:51.419738 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:51.419775 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:56:51.420348 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:51.420358 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:51.420425 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:51.420589 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:51.420943 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:51.421878 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:51.422760 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:51.423323 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:51.423432 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:51.424186 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:51.424190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:51.424190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:51.424418 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:51.424462 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:51.424479 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:51.424782 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:51.424856 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:56:51.424885 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":115,"threads_started":1}
I20260812 03:56:51.425066 9720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d57a6e29f6b4394a337b30e26ea2dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d57a6e29f6b4394a337b30e26ea2dc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35141 } } }
I20260812 03:56:51.425493 9720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:51.425228 9719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:51.425980 9728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:51.426523 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:51.426698 9728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:51.428213 9728 catalog_manager.cc:1347] Generated new cluster ID: 8d292358cc1942dabd350c8c9af56dc7
I20260812 03:56:51.428265 9728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:51.431913 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:51.428102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:39481 (local address 127.0.0.1:35141)
0812 03:56:51.428141 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:51.428143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:51.428231 (+ 88us) server_negotiation.cc:407] Connection header received
0812 03:56:51.428337 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:51.428338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:51.428350 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:51.428418 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:51.428585 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:51.429331 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:51.430436 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:51.430757 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:51.430835 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:51.431644 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:51.431646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:51.431646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:51.431650 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:51.431666 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:51.431671 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:51.431729 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:51.431829 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:56:51.431839 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":20}
I20260812 03:56:51.438647 9728 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:51.439369 9728 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:51.443253 9728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d57a6e29f6b4394a337b30e26ea2dc5: Generated new TSK 0
I20260812 03:56:51.443456 9728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:51.449023 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 9634
W20260812 03:56:51.457170 9616 connection.cc:535] client connection to 127.0.0.1:35141 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35141): Connection reset by peer (error 104)
2026-08-12T03:56:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:51 dist-test-slave-tm1g krb5kdc[9746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:51 dist-test-slave-tm1g krb5kdc[9746](info): set up 2 sockets
Aug 12 03:56:51 dist-test-slave-tm1g krb5kdc[9746](info): commencing operation
krb5kdc: starting...
W20260812 03:56:52.578064 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:52 dist-test-slave-tm1g krb5kdc[9746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:56:52.714133 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40073
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:52.799049 9763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:52.799265 9763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:52.799288 9763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:52.800434 9763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:52.800482 9763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:52.800495 9763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:52.800508 9763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:52.800520 9763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:52.801931 9763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40073
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:52.802134 9763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:52.802326 9763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:52.807327 9769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:52.807466 9763 server_base.cc:1034] running on GCE node
W20260812 03:56:52.807292 9770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:52.807318 9773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:52.807785 9763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:52.808089 9763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:52.809248 9763 hybrid_clock.cc:648] HybridClock initialized: now 1786507012809234 us; error 29 us; skew 500 ppm
Aug 12 03:56:52 dist-test-slave-tm1g krb5kdc[9746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507012, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:52.811954 9763 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:56:52.812456 9763 webserver.cc:460] Webserver started at http://127.0.0.1:46859/ using document root <none> and password file <none>
I20260812 03:56:52.812695 9763 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:52.812767 9763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:52.812927 9763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:52.814073 9763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "a3405656e6644f3990ca8f9b0216ac86"
format_stamp: "Formatted at 2026-08-12 03:56:52 on dist-test-slave-tm1g"
server_key: "a37d6568063f63c7bc066d41df6753ba"
server_key_iv: "db5ccb1c54c858cdf272f6980286cbca"
server_key_version: "encryptionkey@0"
I20260812 03:56:52.814486 9763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "a3405656e6644f3990ca8f9b0216ac86"
format_stamp: "Formatted at 2026-08-12 03:56:52 on dist-test-slave-tm1g"
server_key: "a37d6568063f63c7bc066d41df6753ba"
server_key_iv: "db5ccb1c54c858cdf272f6980286cbca"
server_key_version: "encryptionkey@0"
I20260812 03:56:52.815990 9763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:52.816933 9781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:52.817150 9763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:52.817263 9763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "a3405656e6644f3990ca8f9b0216ac86"
format_stamp: "Formatted at 2026-08-12 03:56:52 on dist-test-slave-tm1g"
server_key: "a37d6568063f63c7bc066d41df6753ba"
server_key_iv: "db5ccb1c54c858cdf272f6980286cbca"
server_key_version: "encryptionkey@0"
I20260812 03:56:52.817344 9763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:52.820863 9763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:52.823071 9763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:52.823236 9763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:52.827500 9763 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40073
I20260812 03:56:52.827535 9838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40073 every 8 connection(s)
I20260812 03:56:52.827907 9763 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:52.828819 9839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:52.830152 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 9763
I20260812 03:56:52.830271 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:52.830531 17627 external_mini_cluster.cc:1428] Setting key 89574f422c1549ed962c476bf54d7990
I20260812 03:56:52.831736 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86: Bootstrap starting.
I20260812 03:56:52.832578 9839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:52.832933 9839 log.cc:824] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:52.833482 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:52.831119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:49873 (local address 127.0.0.1:40073)
0812 03:56:52.831313 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:52.831320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:52.831345 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:56:52.831774 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:52.831803 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:52.831849 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:52.831984 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:52.832817 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:52.832820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:52.832821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:52.832825 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:52.832882 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:52.832893 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:52.833250 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:52.833289 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 03:56:52.833329 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260812 03:56:52.833798 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86: No bootstrap required, opened a new log
I20260812 03:56:52.834733 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:52.836820 9839 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3405656e6644f3990ca8f9b0216ac86" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40073 } }
I20260812 03:56:52.837031 9839 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:52.837076 9839 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3405656e6644f3990ca8f9b0216ac86, State: Initialized, Role: FOLLOWER
I20260812 03:56:52.837257 9839 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3405656e6644f3990ca8f9b0216ac86" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40073 } }
I20260812 03:56:52.837378 9839 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:52.837424 9839 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:52.837479 9839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:52.837615 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:52.836462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:60653 (local address 127.0.0.1:40073)
0812 03:56:52.836523 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:52.836525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:52.836620 (+ 95us) server_negotiation.cc:407] Connection header received
0812 03:56:52.836672 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:52.836673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:52.836679 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:52.836733 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:52.837326 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:52.837327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:52.837327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:52.837332 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:52.837349 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:52.837354 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:52.837416 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:52.837522 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 03:56:52.837538 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:56:52.838138 9839 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3405656e6644f3990ca8f9b0216ac86" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40073 } }
I20260812 03:56:52.838227 9839 leader_election.cc:302] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3405656e6644f3990ca8f9b0216ac86; no voters:
I20260812 03:56:52.838411 9839 leader_election.cc:288] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:52.838438 9853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:52.838603 9853 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [term 1 LEADER]: Becoming Leader. State: Replica: a3405656e6644f3990ca8f9b0216ac86, State: Running, Role: LEADER
I20260812 03:56:52.838676 9839 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:52.838737 9853 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3405656e6644f3990ca8f9b0216ac86" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40073 } }
I20260812 03:56:52.839178 9855 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3405656e6644f3990ca8f9b0216ac86. Latest consensus state: current_term: 1 leader_uuid: "a3405656e6644f3990ca8f9b0216ac86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3405656e6644f3990ca8f9b0216ac86" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40073 } } }
I20260812 03:56:52.839155 9854 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3405656e6644f3990ca8f9b0216ac86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3405656e6644f3990ca8f9b0216ac86" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40073 } } }
I20260812 03:56:52.839381 9854 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:52.839367 9855 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:52.840050 9864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:52.840544 9864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:52.841984 9864 catalog_manager.cc:1347] Generated new cluster ID: e8f79c5375e5448180936f82c2b79e43
I20260812 03:56:52.842064 9864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:52.857414 9864 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:52.857651 9864 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:52.862015 9864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3405656e6644f3990ca8f9b0216ac86: Generated new TSK 0
I20260812 03:56:52.862257 9864 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:52.868876 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 9763
2026-08-12T03:56:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:52 dist-test-slave-tm1g krb5kdc[9878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:52 dist-test-slave-tm1g krb5kdc[9878](info): set up 2 sockets
Aug 12 03:56:52 dist-test-slave-tm1g krb5kdc[9878](info): commencing operation
krb5kdc: starting...
W20260812 03:56:54.014137 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:54 dist-test-slave-tm1g krb5kdc[9878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:56:54.149313 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39323
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:36371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:39323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:54.232939 9893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:54.233219 9893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:54.233261 9893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:54.234529 9893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:54.234611 9893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:54.234637 9893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:54.234658 9893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:54.234689 9893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:54.236294 9893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:36371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:39323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:54.236596 9893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:54.236842 9893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:54.242203 9901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:54.242165 9903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:54.242173 9900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:54.242233 9893 server_base.cc:1034] running on GCE node
I20260812 03:56:54.242774 9893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:54.243198 9893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:54.244393 9893 hybrid_clock.cc:648] HybridClock initialized: now 1786507014244378 us; error 30 us; skew 500 ppm
Aug 12 03:56:54 dist-test-slave-tm1g krb5kdc[9878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507014, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:54.251704 9893 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:54.252243 9893 webserver.cc:460] Webserver started at http://127.17.54.254:43359/ using document root <none> and password file <none>
I20260812 03:56:54.252493 9893 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:54.252566 9893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:54.252723 9893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:54.253845 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "87da354e95a94704b213567766bb8892"
format_stamp: "Formatted at 2026-08-12 03:56:54 on dist-test-slave-tm1g"
server_key: "263a002ec76cddd20b6761f8333406b3"
server_key_iv: "51e2d72073aad250a0817d50714025c6"
server_key_version: "encryptionkey@0"
I20260812 03:56:54.254236 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "87da354e95a94704b213567766bb8892"
format_stamp: "Formatted at 2026-08-12 03:56:54 on dist-test-slave-tm1g"
server_key: "263a002ec76cddd20b6761f8333406b3"
server_key_iv: "51e2d72073aad250a0817d50714025c6"
server_key_version: "encryptionkey@0"
I20260812 03:56:54.255692 9893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:54.256572 9911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:54.256850 9893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:56:54.256987 9893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "87da354e95a94704b213567766bb8892"
format_stamp: "Formatted at 2026-08-12 03:56:54 on dist-test-slave-tm1g"
server_key: "263a002ec76cddd20b6761f8333406b3"
server_key_iv: "51e2d72073aad250a0817d50714025c6"
server_key_version: "encryptionkey@0"
I20260812 03:56:54.257083 9893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:54.267700 9893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:54.270162 9893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:54.270406 9893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:54.274924 9893 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:39323
I20260812 03:56:54.275060 9973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:39323 every 8 connection(s)
I20260812 03:56:54.275414 9893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:54.276435 9974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:54.279413 9974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892: Bootstrap starting.
I20260812 03:56:54.280268 9974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:54.280608 9974 log.cc:824] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:54.281422 9974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892: No bootstrap required, opened a new log
I20260812 03:56:54.283336 9974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87da354e95a94704b213567766bb8892" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39323 } }
I20260812 03:56:54.283497 9974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:54.283524 9974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87da354e95a94704b213567766bb8892, State: Initialized, Role: FOLLOWER
I20260812 03:56:54.283751 9974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87da354e95a94704b213567766bb8892" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39323 } }
I20260812 03:56:54.283834 9974 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:54.283890 9974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:54.283950 9974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:54.284952 9974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87da354e95a94704b213567766bb8892" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39323 } }
I20260812 03:56:54.285077 9974 leader_election.cc:302] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87da354e95a94704b213567766bb8892; no voters:
I20260812 03:56:54.285346 9974 leader_election.cc:288] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:54.285395 9977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:54.285477 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 9893
I20260812 03:56:54.285617 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:54.285605 9977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [term 1 LEADER]: Becoming Leader. State: Replica: 87da354e95a94704b213567766bb8892, State: Running, Role: LEADER
I20260812 03:56:54.285699 9974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:54.285815 17627 external_mini_cluster.cc:1428] Setting key 0c102a04ed46f7f8214d4bd2191e2c99
I20260812 03:56:54.285691 9977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87da354e95a94704b213567766bb8892" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39323 } }
I20260812 03:56:54.287698 9979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87da354e95a94704b213567766bb8892. Latest consensus state: current_term: 1 leader_uuid: "87da354e95a94704b213567766bb8892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87da354e95a94704b213567766bb8892" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39323 } } }
I20260812 03:56:54.287971 9979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:54.287734 9978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87da354e95a94704b213567766bb8892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87da354e95a94704b213567766bb8892" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39323 } } }
W20260812 03:56:54.288064 9998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 03:56:54.288127 9978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892 [sys.catalog]: This master's current role is: LEADER
W20260812 03:56:54.288146 9998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:56:54.288543 10000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:54.288776 10000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:54.293756 10000 catalog_manager.cc:1347] Generated new cluster ID: 4c4b4e222a2d46ee8c3de743329a650d
I20260812 03:56:54.293834 10000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:56:54 dist-test-slave-tm1g krb5kdc[9878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:54.299662 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:54.286391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58511 (local address 127.17.54.254:39323)
0812 03:56:54.287361 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:54.287373 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:54.287415 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:56:54.288037 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:54.288048 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:54.288120 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:54.288327 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:54.289115 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:54.290425 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:54.291515 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:54.291994 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:54.292084 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:54.295148 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:54.295153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:54.295153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:54.295167 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:54.297379 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:54.297874 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:54.297878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:54.297878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:54.298010 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:54.298251 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:54.298252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:54.298252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:54.298612 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:54.298763 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:54.299288 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 03:56:54.299414 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":90,"threads_started":1}
I20260812 03:56:54.300987 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:54.314275 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:54.302669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47765 (local address 127.17.54.254:39323)
0812 03:56:54.302708 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:54.302710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:54.306539 (+ 3829us) server_negotiation.cc:407] Connection header received
0812 03:56:54.306599 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:54.306600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:54.306611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:54.306676 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:54.307349 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:54.308215 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:54.309286 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:54.309637 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:54.309700 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:54.311272 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:54.311273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:54.311273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:54.311282 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:54.312942 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:54.313390 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:54.313391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:54.313391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:54.313454 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:54.313696 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:54.313697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:54.313697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:54.313825 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:54.313923 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:54.314095 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:56:54.314165 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:56:54.316247 10000 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:54.317008 10000 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:54.321310 10000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87da354e95a94704b213567766bb8892: Generated new TSK 0
I20260812 03:56:54.321544 10000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:54.323777 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 9893
W20260812 03:56:54.333184 9875 connection.cc:535] client connection to 127.17.54.254:39323 recv error: Network error: failed to read from TLS socket (remote: 127.17.54.254:39323): Connection reset by peer (error 104)
2026-08-12T03:56:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:54 dist-test-slave-tm1g krb5kdc[10009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:54 dist-test-slave-tm1g krb5kdc[10009](info): set up 2 sockets
Aug 12 03:56:54 dist-test-slave-tm1g krb5kdc[10009](info): commencing operation
krb5kdc: starting...
W20260812 03:56:55.442396 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:55 dist-test-slave-tm1g krb5kdc[10009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:56:55.571496 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:45033
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:37691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:45033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:55.656348 10025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:55.656574 10025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:55.656630 10025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:55.657783 10025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:55.657824 10025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:55.657840 10025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:55.657852 10025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:55.657864 10025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:55.659338 10025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:37691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:45033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:55.659547 10025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:55.659721 10025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:56:55.665203 10025 server_base.cc:1034] running on GCE node
W20260812 03:56:55.665184 10031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:55.665184 10035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:55.665170 10032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:55.665766 10025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:55.666085 10025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:55.667259 10025 hybrid_clock.cc:648] HybridClock initialized: now 1786507015667262 us; error 47 us; skew 500 ppm
Aug 12 03:56:55 dist-test-slave-tm1g krb5kdc[10009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507015, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:55.669818 10025 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:55.670415 10025 webserver.cc:460] Webserver started at http://127.17.54.254:39221/ using document root <none> and password file <none>
I20260812 03:56:55.670665 10025 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:55.670735 10025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:55.670897 10025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:55.672013 10025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "39bc84ac81224cabb7eb805ccfa6b46b"
format_stamp: "Formatted at 2026-08-12 03:56:55 on dist-test-slave-tm1g"
server_key: "8ea8dc5cfbdc5bd33a87e700477cd769"
server_key_iv: "ea661f94b47f1e5ff6957eea74e508c7"
server_key_version: "encryptionkey@0"
I20260812 03:56:55.672396 10025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "39bc84ac81224cabb7eb805ccfa6b46b"
format_stamp: "Formatted at 2026-08-12 03:56:55 on dist-test-slave-tm1g"
server_key: "8ea8dc5cfbdc5bd33a87e700477cd769"
server_key_iv: "ea661f94b47f1e5ff6957eea74e508c7"
server_key_version: "encryptionkey@0"
I20260812 03:56:55.673857 10025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:55.674759 10042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:55.674929 10025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:56:55.675055 10025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "39bc84ac81224cabb7eb805ccfa6b46b"
format_stamp: "Formatted at 2026-08-12 03:56:55 on dist-test-slave-tm1g"
server_key: "8ea8dc5cfbdc5bd33a87e700477cd769"
server_key_iv: "ea661f94b47f1e5ff6957eea74e508c7"
server_key_version: "encryptionkey@0"
I20260812 03:56:55.675134 10025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:55.690102 10025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:55.692430 10025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:55.692659 10025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:55.697238 10025 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:45033
I20260812 03:56:55.697268 10109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:45033 every 8 connection(s)
I20260812 03:56:55.697656 10025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:55.698608 10110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:55.700915 10110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b: Bootstrap starting.
I20260812 03:56:55.701519 10110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:55.701828 10110 log.cc:824] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:55.702589 10110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b: No bootstrap required, opened a new log
I20260812 03:56:55.704196 10110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bc84ac81224cabb7eb805ccfa6b46b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45033 } }
I20260812 03:56:55.704370 10110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:55.704437 10110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39bc84ac81224cabb7eb805ccfa6b46b, State: Initialized, Role: FOLLOWER
I20260812 03:56:55.704597 10110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bc84ac81224cabb7eb805ccfa6b46b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45033 } }
I20260812 03:56:55.704675 10110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:55.704720 10110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:55.704775 10110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:55.705642 10110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bc84ac81224cabb7eb805ccfa6b46b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45033 } }
I20260812 03:56:55.705760 10110 leader_election.cc:302] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39bc84ac81224cabb7eb805ccfa6b46b; no voters:
I20260812 03:56:55.705935 10110 leader_election.cc:288] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:55.705979 10114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:55.706161 10114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [term 1 LEADER]: Becoming Leader. State: Replica: 39bc84ac81224cabb7eb805ccfa6b46b, State: Running, Role: LEADER
I20260812 03:56:55.706192 10110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:55.706255 10114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bc84ac81224cabb7eb805ccfa6b46b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45033 } }
I20260812 03:56:55.706765 10115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39bc84ac81224cabb7eb805ccfa6b46b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bc84ac81224cabb7eb805ccfa6b46b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45033 } } }
I20260812 03:56:55.706869 10115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:55.706768 10117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39bc84ac81224cabb7eb805ccfa6b46b. Latest consensus state: current_term: 1 leader_uuid: "39bc84ac81224cabb7eb805ccfa6b46b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bc84ac81224cabb7eb805ccfa6b46b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 45033 } } }
I20260812 03:56:55.707018 10117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:55.707118 10125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:55.707502 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 10025
I20260812 03:56:55.707645 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:55.707687 10125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:55.707866 17627 external_mini_cluster.cc:1428] Setting key a482f676d1f671f910adcd2a6d56fd43
I20260812 03:56:55.709477 10125 catalog_manager.cc:1347] Generated new cluster ID: e6923d04510344c99f1c81f5ebaed775
I20260812 03:56:55.709537 10125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:55.713219 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:55.708355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34211 (local address 127.17.54.254:45033)
0812 03:56:55.708564 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:55.708573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:55.708622 (+ 49us) server_negotiation.cc:407] Connection header received
0812 03:56:55.708735 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:55.708743 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:55.708798 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:55.708950 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:55.709318 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:55.710415 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:55.711250 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:55.711684 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:55.711800 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:55.712566 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:55.712569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:55.712570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:55.712750 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:55.712780 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:55.712791 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:55.712877 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:55.712960 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:56:55.712990 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":74,"threads_started":1}
I20260812 03:56:55.714141 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:55.717844 10125 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:55.718596 10125 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:55.718596 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:55.715552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59827 (local address 127.17.54.254:45033)
0812 03:56:55.715590 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:55.715592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:55.715692 (+ 100us) server_negotiation.cc:407] Connection header received
0812 03:56:55.715751 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:55.715752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:55.715762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:55.715818 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:55.715973 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:55.716450 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:55.717155 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:55.717411 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:55.717481 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:55.718219 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:55.718220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:55.718220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:55.718225 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:55.718240 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:55.718244 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:55.718322 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:55.718406 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:56:55.718525 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":18}
I20260812 03:56:55.725623 10125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39bc84ac81224cabb7eb805ccfa6b46b: Generated new TSK 0
I20260812 03:56:55.725855 10125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:55.728902 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 10025
W20260812 03:56:55.738883 10007 connection.cc:535] client connection to 127.17.54.254:45033 recv error: Network error: failed to read from TLS socket (remote: 127.17.54.254:45033): Connection reset by peer (error 104)
2026-08-12T03:56:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:55 dist-test-slave-tm1g krb5kdc[10148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:55 dist-test-slave-tm1g krb5kdc[10148](info): set up 2 sockets
Aug 12 03:56:55 dist-test-slave-tm1g krb5kdc[10148](info): commencing operation
krb5kdc: starting...
W20260812 03:56:56.856312 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:56 dist-test-slave-tm1g krb5kdc[10148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:56:56.990276 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42771
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:46573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:42771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:57.075874 10163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:57.076097 10163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:57.076121 10163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:57.077358 10163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:57.077410 10163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:57.077425 10163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:57.077438 10163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:57.077450 10163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:57.079061 10163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:46573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:42771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:42771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:57.079388 10163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:57.079651 10163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:57.084980 10169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:57.085146 10168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:57.085186 10163 server_base.cc:1034] running on GCE node
W20260812 03:56:57.084996 10171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:57.085685 10163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:57.086030 10163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:57.087218 10163 hybrid_clock.cc:648] HybridClock initialized: now 1786507017087172 us; error 68 us; skew 500 ppm
Aug 12 03:56:57 dist-test-slave-tm1g krb5kdc[10148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:57.090036 10163 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:57.090596 10163 webserver.cc:460] Webserver started at http://127.17.54.254:43183/ using document root <none> and password file <none>
I20260812 03:56:57.090843 10163 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:57.090919 10163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:57.091073 10163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:57.092186 10163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "6bb36ec75b1847d084dca162fca57eeb"
format_stamp: "Formatted at 2026-08-12 03:56:57 on dist-test-slave-tm1g"
server_key: "f223f15ffedc66a93c8c33b130848e1f"
server_key_iv: "8e18f997884d1d61ed4365f2c766c7ad"
server_key_version: "encryptionkey@0"
I20260812 03:56:57.092573 10163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "6bb36ec75b1847d084dca162fca57eeb"
format_stamp: "Formatted at 2026-08-12 03:56:57 on dist-test-slave-tm1g"
server_key: "f223f15ffedc66a93c8c33b130848e1f"
server_key_iv: "8e18f997884d1d61ed4365f2c766c7ad"
server_key_version: "encryptionkey@0"
I20260812 03:56:57.094048 10163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:56:57.095016 10179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:57.095388 10163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:56:57.095510 10163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "6bb36ec75b1847d084dca162fca57eeb"
format_stamp: "Formatted at 2026-08-12 03:56:57 on dist-test-slave-tm1g"
server_key: "f223f15ffedc66a93c8c33b130848e1f"
server_key_iv: "8e18f997884d1d61ed4365f2c766c7ad"
server_key_version: "encryptionkey@0"
I20260812 03:56:57.095592 10163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:57.099002 10163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:57.101233 10163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:57.101403 10163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:57.105633 10163 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:42771
I20260812 03:56:57.105708 10245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:42771 every 8 connection(s)
I20260812 03:56:57.106057 10163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:57.106416 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 10163
I20260812 03:56:57.106546 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:57.106805 17627 external_mini_cluster.cc:1428] Setting key d809db75d4f64c8316a6199b1aaea435
I20260812 03:56:57.107372 10246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:57.109587 10250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:57.107386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41113 (local address 127.17.54.254:42771)
0812 03:56:57.107568 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:57.107578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:57.107609 (+ 31us) server_negotiation.cc:407] Connection header received
0812 03:56:57.108175 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:57.108184 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:57.108217 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:57.108427 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:57.109147 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:57.109151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:57.109151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:57.109157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:57.109203 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:57.109218 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:57.109304 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:57.109346 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 03:56:57.109386 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260812 03:56:57.110630 10246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb: Bootstrap starting.
I20260812 03:56:57.110713 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:57.111384 10246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:57.111727 10246 log.cc:824] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:57.112571 10246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb: No bootstrap required, opened a new log
I20260812 03:56:57.113741 10250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:57.112414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38621 (local address 127.17.54.254:42771)
0812 03:56:57.112475 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:57.112477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:57.112575 (+ 98us) server_negotiation.cc:407] Connection header received
0812 03:56:57.112800 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:57.112802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:57.112807 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:57.112872 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:57.113520 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:57.113521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:57.113521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:57.113526 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:57.113543 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:57.113549 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:57.113613 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:57.113668 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:56:57.113682 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:56:57.115198 10246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb36ec75b1847d084dca162fca57eeb" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42771 } }
I20260812 03:56:57.115345 10246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:57.115370 10246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb36ec75b1847d084dca162fca57eeb, State: Initialized, Role: FOLLOWER
I20260812 03:56:57.115495 10246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb36ec75b1847d084dca162fca57eeb" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42771 } }
I20260812 03:56:57.115597 10246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:57.115622 10246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:57.115653 10246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:57.116226 10246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb36ec75b1847d084dca162fca57eeb" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42771 } }
I20260812 03:56:57.116317 10246 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bb36ec75b1847d084dca162fca57eeb; no voters:
I20260812 03:56:57.116495 10246 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:57.116629 10257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:57.116803 10257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [term 1 LEADER]: Becoming Leader. State: Replica: 6bb36ec75b1847d084dca162fca57eeb, State: Running, Role: LEADER
I20260812 03:56:57.116811 10246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:57.116922 10257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb36ec75b1847d084dca162fca57eeb" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42771 } }
I20260812 03:56:57.117355 10258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bb36ec75b1847d084dca162fca57eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb36ec75b1847d084dca162fca57eeb" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42771 } } }
I20260812 03:56:57.118160 10258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:57.118060 10259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb36ec75b1847d084dca162fca57eeb. Latest consensus state: current_term: 1 leader_uuid: "6bb36ec75b1847d084dca162fca57eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb36ec75b1847d084dca162fca57eeb" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 42771 } } }
I20260812 03:56:57.118444 10259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb [sys.catalog]: This master's current role is: LEADER
W20260812 03:56:57.119005 10275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:56:57.119050 10275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:56:57.119114 10273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:57.119264 10273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:57.120759 10273 catalog_manager.cc:1347] Generated new cluster ID: 65cd635f32dd4efbacb21ffdedc96987
I20260812 03:56:57.120815 10273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:57.128001 10273 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:57.128199 10273 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:57.133504 10273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bb36ec75b1847d084dca162fca57eeb: Generated new TSK 0
I20260812 03:56:57.133711 10273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:57.145172 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 10163
2026-08-12T03:56:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:57 dist-test-slave-tm1g krb5kdc[10280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:57 dist-test-slave-tm1g krb5kdc[10280](info): set up 2 sockets
Aug 12 03:56:57 dist-test-slave-tm1g krb5kdc[10280](info): commencing operation
krb5kdc: starting...
W20260812 03:56:58.265846 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:58 dist-test-slave-tm1g krb5kdc[10280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:56:58.402680 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39237
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:35017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:39237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:58.486222 10294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:58.486506 10294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:58.486547 10294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:58.487718 10294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:58.487798 10294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:58.487823 10294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:58.487852 10294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:58.487888 10294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:58.489431 10294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:35017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:39237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:58.489733 10294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:58.489979 10294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:58.495199 10299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:58.495285 10302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:58.495304 10294 server_base.cc:1034] running on GCE node
W20260812 03:56:58.495303 10300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:58.495685 10294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:58.496037 10294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:58.497210 10294 hybrid_clock.cc:648] HybridClock initialized: now 1786507018497184 us; error 38 us; skew 500 ppm
Aug 12 03:56:58 dist-test-slave-tm1g krb5kdc[10280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507018, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:58.499940 10294 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:58.500439 10294 webserver.cc:460] Webserver started at http://127.17.54.254:45845/ using document root <none> and password file <none>
I20260812 03:56:58.500654 10294 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:58.500705 10294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:58.500798 10294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:58.501856 10294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "5369ef3d30394855ab0744a94ecbe6c9"
format_stamp: "Formatted at 2026-08-12 03:56:58 on dist-test-slave-tm1g"
server_key: "7f763e9e66dd6650ab88a00e5d869b9b"
server_key_iv: "709780226e8e0c846b251422c9cd92d8"
server_key_version: "encryptionkey@0"
I20260812 03:56:58.502190 10294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "5369ef3d30394855ab0744a94ecbe6c9"
format_stamp: "Formatted at 2026-08-12 03:56:58 on dist-test-slave-tm1g"
server_key: "7f763e9e66dd6650ab88a00e5d869b9b"
server_key_iv: "709780226e8e0c846b251422c9cd92d8"
server_key_version: "encryptionkey@0"
I20260812 03:56:58.503784 10294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:58.504631 10309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:58.504834 10294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:58.504961 10294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "5369ef3d30394855ab0744a94ecbe6c9"
format_stamp: "Formatted at 2026-08-12 03:56:58 on dist-test-slave-tm1g"
server_key: "7f763e9e66dd6650ab88a00e5d869b9b"
server_key_iv: "709780226e8e0c846b251422c9cd92d8"
server_key_version: "encryptionkey@0"
I20260812 03:56:58.505044 10294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:58.512603 10294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:58.514772 10294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:58.514958 10294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:58.519462 10294 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:39237
I20260812 03:56:58.519503 10369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:39237 every 8 connection(s)
I20260812 03:56:58.519888 10294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:58.520807 10370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:58.523204 10370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9: Bootstrap starting.
I20260812 03:56:58.523837 10370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:58.524139 10370 log.cc:824] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:58.524899 10370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9: No bootstrap required, opened a new log
I20260812 03:56:58.526687 10370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5369ef3d30394855ab0744a94ecbe6c9" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39237 } }
I20260812 03:56:58.526829 10370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:58.526854 10370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5369ef3d30394855ab0744a94ecbe6c9, State: Initialized, Role: FOLLOWER
I20260812 03:56:58.527027 10370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5369ef3d30394855ab0744a94ecbe6c9" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39237 } }
I20260812 03:56:58.527094 10370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:58.527117 10370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:58.527164 10370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:58.528061 10370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5369ef3d30394855ab0744a94ecbe6c9" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39237 } }
I20260812 03:56:58.528152 10370 leader_election.cc:302] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5369ef3d30394855ab0744a94ecbe6c9; no voters:
I20260812 03:56:58.528339 10370 leader_election.cc:288] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:58.528486 10373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:58.528628 10370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:58.528656 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 10294
I20260812 03:56:58.528678 10373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [term 1 LEADER]: Becoming Leader. State: Replica: 5369ef3d30394855ab0744a94ecbe6c9, State: Running, Role: LEADER
I20260812 03:56:58.528780 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:58.528760 10373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5369ef3d30394855ab0744a94ecbe6c9" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39237 } }
I20260812 03:56:58.528998 17627 external_mini_cluster.cc:1428] Setting key 555c14b44cf74c7a81a28a2477acb1b1
I20260812 03:56:58.529196 10373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5369ef3d30394855ab0744a94ecbe6c9. Latest consensus state: current_term: 1 leader_uuid: "5369ef3d30394855ab0744a94ecbe6c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5369ef3d30394855ab0744a94ecbe6c9" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39237 } } }
I20260812 03:56:58.529297 10373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:58.529353 10375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5369ef3d30394855ab0744a94ecbe6c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5369ef3d30394855ab0744a94ecbe6c9" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39237 } } }
I20260812 03:56:58.529421 10375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9 [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:58.529932 10380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:58.530566 10380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:58.532318 10380 catalog_manager.cc:1347] Generated new cluster ID: ba24ca172651475497ae6f36c44fc767
I20260812 03:56:58.532371 10380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:56:58 dist-test-slave-tm1g krb5kdc[10280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:56:58.541400 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:58.529729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39239 (local address 127.17.54.254:39237)
0812 03:56:58.529917 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:58.529928 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:58.529961 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:56:58.530088 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:58.530095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:58.530154 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:58.530924 (+ 770us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:58.531169 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:58.532514 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:58.533589 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:58.534192 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:58.534294 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:58.537277 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:58.537281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:58.537282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:58.537297 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:58.539563 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:58.540031 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:58.540035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:58.540036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:58.540171 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:58.540389 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:58.540389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:58.540390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:58.540752 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:58.540923 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:58.541011 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:56:58.541162 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260812 03:56:58.542583 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:58.545189 10380 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:56:58.545888 10380 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:56:58.551684 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:58.544179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38149 (local address 127.17.54.254:39237)
0812 03:56:58.544223 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:58.544225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:58.544351 (+ 126us) server_negotiation.cc:407] Connection header received
0812 03:56:58.544399 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:58.544400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:58.544411 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:58.544488 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:58.545218 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:58.545938 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:58.546958 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:58.547306 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:58.547383 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:58.548789 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:58.548791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:58.548791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:56:58.548800 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:58.550506 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:58.550888 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:58.550889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:58.550890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:58.550950 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:56:58.551148 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:56:58.551149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:56:58.551149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:56:58.551286 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:58.551390 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:58.551491 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:56:58.551587 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:56:58.554193 10380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5369ef3d30394855ab0744a94ecbe6c9: Generated new TSK 0
I20260812 03:56:58.554432 10380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:56:58.556670 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 10294
2026-08-12T03:56:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:56:58 dist-test-slave-tm1g krb5kdc[10405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:56:58 dist-test-slave-tm1g krb5kdc[10405](info): set up 2 sockets
Aug 12 03:56:58 dist-test-slave-tm1g krb5kdc[10405](info): commencing operation
krb5kdc: starting...
W20260812 03:56:59.677579 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:56:59 dist-test-slave-tm1g krb5kdc[10405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:56:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:56:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:56:59.813781 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:38543
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:38551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:38543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:56:59.899439 10419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:56:59.899690 10419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:56:59.899727 10419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:56:59.900894 10419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:56:59.900964 10419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:56:59.900990 10419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:56:59.901018 10419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:56:59.901051 10419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:56:59.902634 10419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:38551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:38543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:38543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:56:59.902927 10419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:56:59.903165 10419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:56:59.908639 10425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:59.908730 10426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:56:59.908643 10428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:56:59.909101 10419 server_base.cc:1034] running on GCE node
I20260812 03:56:59.909373 10419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:56:59.909669 10419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:56:59.910840 10419 hybrid_clock.cc:648] HybridClock initialized: now 1786507019910827 us; error 32 us; skew 500 ppm
Aug 12 03:56:59 dist-test-slave-tm1g krb5kdc[10405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507019, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:56:59.917835 10419 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:56:59.918406 10419 webserver.cc:460] Webserver started at http://127.17.54.254:46851/ using document root <none> and password file <none>
I20260812 03:56:59.918646 10419 fs_manager.cc:359] Metadata directory not provided
I20260812 03:56:59.918716 10419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:56:59.918887 10419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:56:59.920421 10419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "f93e8dd166424aff861d3759774bdc6b"
format_stamp: "Formatted at 2026-08-12 03:56:59 on dist-test-slave-tm1g"
server_key: "f84616f60b6d0bf0f09bb86194e1ee92"
server_key_iv: "b130e7d3422c4305ab8d8af6aa75f627"
server_key_version: "encryptionkey@0"
I20260812 03:56:59.920799 10419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "f93e8dd166424aff861d3759774bdc6b"
format_stamp: "Formatted at 2026-08-12 03:56:59 on dist-test-slave-tm1g"
server_key: "f84616f60b6d0bf0f09bb86194e1ee92"
server_key_iv: "b130e7d3422c4305ab8d8af6aa75f627"
server_key_version: "encryptionkey@0"
I20260812 03:56:59.922194 10419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 03:56:59.923125 10437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:56:59.923420 10419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:56:59.923554 10419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "f93e8dd166424aff861d3759774bdc6b"
format_stamp: "Formatted at 2026-08-12 03:56:59 on dist-test-slave-tm1g"
server_key: "f84616f60b6d0bf0f09bb86194e1ee92"
server_key_iv: "b130e7d3422c4305ab8d8af6aa75f627"
server_key_version: "encryptionkey@0"
I20260812 03:56:59.923635 10419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:56:59.967924 10419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:56:59.970391 10419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:56:59.970569 10419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:56:59.974629 10419 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:38543
I20260812 03:56:59.974681 10500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:38543 every 8 connection(s)
I20260812 03:56:59.975059 10419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:56:59.976045 10501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:56:59.978370 10501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b: Bootstrap starting.
I20260812 03:56:59.978981 10501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b: Neither blocks nor log segments found. Creating new log.
I20260812 03:56:59.979235 10501 log.cc:824] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:56:59.979924 10501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b: No bootstrap required, opened a new log
I20260812 03:56:59.980103 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 10419
I20260812 03:56:59.980247 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:56:59.980459 17627 external_mini_cluster.cc:1428] Setting key d26c3cdc214721dadab1924bbecbc4b8
I20260812 03:56:59.982268 10501 raft_consensus.cc:348] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f93e8dd166424aff861d3759774bdc6b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38543 } }
I20260812 03:56:59.982422 10501 raft_consensus.cc:374] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:56:59.982445 10501 raft_consensus.cc:729] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f93e8dd166424aff861d3759774bdc6b, State: Initialized, Role: FOLLOWER
I20260812 03:56:59.982590 10501 consensus_queue.cc:260] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f93e8dd166424aff861d3759774bdc6b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38543 } }
I20260812 03:56:59.982651 10501 raft_consensus.cc:388] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:56:59.982672 10501 raft_consensus.cc:482] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:56:59.982738 10501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:56:59.983314 10501 raft_consensus.cc:504] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f93e8dd166424aff861d3759774bdc6b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38543 } }
I20260812 03:56:59.983399 10501 leader_election.cc:302] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f93e8dd166424aff861d3759774bdc6b; no voters:
I20260812 03:56:59.983623 10501 leader_election.cc:288] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:56:59.983644 10510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:56:59.983779 10510 raft_consensus.cc:686] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [term 1 LEADER]: Becoming Leader. State: Replica: f93e8dd166424aff861d3759774bdc6b, State: Running, Role: LEADER
I20260812 03:56:59.983865 10510 consensus_queue.cc:237] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f93e8dd166424aff861d3759774bdc6b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38543 } }
I20260812 03:56:59.983892 10501 sys_catalog.cc:564] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:56:59.984908 10511 sys_catalog.cc:455] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f93e8dd166424aff861d3759774bdc6b. Latest consensus state: current_term: 1 leader_uuid: "f93e8dd166424aff861d3759774bdc6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f93e8dd166424aff861d3759774bdc6b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38543 } } }
I20260812 03:56:59.984992 10511 sys_catalog.cc:458] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [sys.catalog]: This master's current role is: LEADER
I20260812 03:56:59.985097 10512 sys_catalog.cc:455] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f93e8dd166424aff861d3759774bdc6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f93e8dd166424aff861d3759774bdc6b" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 38543 } } }
I20260812 03:56:59.985184 10512 sys_catalog.cc:458] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b [sys.catalog]: This master's current role is: LEADER
W20260812 03:56:59.985852 10527 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:56:59.986133 10527 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:56:59.986238 10521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:56:59.986495 10521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:56:59.987098 10508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:59.980998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52959 (local address 127.17.54.254:38543)
0812 03:56:59.981187 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:59.981194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:59.981222 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:56:59.981760 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:59.981766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:59.981818 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:59.981929 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:59.982182 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:59.983633 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:59.984681 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:59.985322 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:59.985429 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:59.986363 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:59.986367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:59.986367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:59.986588 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:59.986627 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:59.986647 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:59.986698 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:59.986794 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:56:59.986849 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260812 03:56:59.988062 10521 catalog_manager.cc:1347] Generated new cluster ID: 63b526e71adb4cacb2b1eadd89ae5a07
I20260812 03:56:59.988111 10521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:56:59.988154 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:56:59.993384 10508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:56:59.989712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45507 (local address 127.17.54.254:38543)
0812 03:56:59.989749 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:56:59.989751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:56:59.989867 (+ 116us) server_negotiation.cc:407] Connection header received
0812 03:56:59.989960 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:56:59.989961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:56:59.989971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:56:59.989999 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:56:59.990199 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:59.990987 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:59.992029 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:56:59.992331 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:56:59.992371 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:56:59.993127 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:56:59.993128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:56:59.993129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:56:59.993133 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:56:59.993148 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:56:59.993152 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:56:59.993179 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 03:56:59.993282 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 03:56:59.993305 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:57:00.000191 10521 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:00.000949 10521 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:00.006639 10521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f93e8dd166424aff861d3759774bdc6b: Generated new TSK 0
I20260812 03:57:00.006863 10521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:57:00.013101 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 10419
2026-08-12T03:57:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:00 dist-test-slave-tm1g krb5kdc[10542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:00 dist-test-slave-tm1g krb5kdc[10542](info): set up 2 sockets
Aug 12 03:57:00 dist-test-slave-tm1g krb5kdc[10542](info): commencing operation
krb5kdc: starting...
W20260812 03:57:01.137415 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:57:01 dist-test-slave-tm1g krb5kdc[10542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:57:01.266078 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39325
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:39325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:01.359153 10556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:01.359405 10556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:01.359433 10556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:01.360791 10556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:01.360843 10556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:01.360860 10556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:01.360903 10556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:01.360934 10556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:01.362797 10556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:39325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:39325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:01.363099 10556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:01.363366 10556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:01.369001 10567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:01.368974 10561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:01.369262 10556 server_base.cc:1034] running on GCE node
W20260812 03:57:01.368974 10563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:01.369648 10556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:01.369953 10556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:01.371114 10556 hybrid_clock.cc:648] HybridClock initialized: now 1786507021371100 us; error 31 us; skew 500 ppm
Aug 12 03:57:01 dist-test-slave-tm1g krb5kdc[10542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507021, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:01.373764 10556 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:57:01.374221 10556 webserver.cc:460] Webserver started at http://127.17.54.254:40323/ using document root <none> and password file <none>
I20260812 03:57:01.374495 10556 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:01.374576 10556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:01.374733 10556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:01.375844 10556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "55a36f8a02de433f901f84c53046dd17"
format_stamp: "Formatted at 2026-08-12 03:57:01 on dist-test-slave-tm1g"
server_key: "30fb453ace46f9a2e7975a887b2198f4"
server_key_iv: "b83a40dd5ba83781049bef9bf26b2401"
server_key_version: "encryptionkey@0"
I20260812 03:57:01.376224 10556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "55a36f8a02de433f901f84c53046dd17"
format_stamp: "Formatted at 2026-08-12 03:57:01 on dist-test-slave-tm1g"
server_key: "30fb453ace46f9a2e7975a887b2198f4"
server_key_iv: "b83a40dd5ba83781049bef9bf26b2401"
server_key_version: "encryptionkey@0"
I20260812 03:57:01.377652 10556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:01.378511 10574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:01.378756 10556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:01.378877 10556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "55a36f8a02de433f901f84c53046dd17"
format_stamp: "Formatted at 2026-08-12 03:57:01 on dist-test-slave-tm1g"
server_key: "30fb453ace46f9a2e7975a887b2198f4"
server_key_iv: "b83a40dd5ba83781049bef9bf26b2401"
server_key_version: "encryptionkey@0"
I20260812 03:57:01.378959 10556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:01.382411 10556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:01.384563 10556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:01.384743 10556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:01.389182 10556 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:39325
I20260812 03:57:01.389216 10633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:39325 every 8 connection(s)
I20260812 03:57:01.389567 10556 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:01.390630 10634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:01.392653 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 10556
I20260812 03:57:01.392773 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:57:01.392974 17627 external_mini_cluster.cc:1428] Setting key 1ad16f10e46cd388cdbd70a2510bb2de
I20260812 03:57:01.393348 10634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17: Bootstrap starting.
I20260812 03:57:01.394375 10634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:01.394728 10634 log.cc:824] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:01.395541 10634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17: No bootstrap required, opened a new log
I20260812 03:57:01.395711 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:01.393519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58843 (local address 127.17.54.254:39325)
0812 03:57:01.393697 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:01.393706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:01.393740 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:57:01.394391 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:01.394401 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:01.394438 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:01.394611 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:01.395267 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:01.395272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:01.395272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:01.395277 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:01.395324 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:01.395338 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:01.395432 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:01.395473 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 03:57:01.395513 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260812 03:57:01.396706 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:57:01.398375 10634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a36f8a02de433f901f84c53046dd17" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39325 } }
I20260812 03:57:01.398589 10634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:57:01.398631 10634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55a36f8a02de433f901f84c53046dd17, State: Initialized, Role: FOLLOWER
I20260812 03:57:01.398759 10634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a36f8a02de433f901f84c53046dd17" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39325 } }
I20260812 03:57:01.398864 10634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:57:01.398902 10634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:57:01.398957 10634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:01.399240 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:01.398081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55485 (local address 127.17.54.254:39325)
0812 03:57:01.398120 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:01.398122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:01.398250 (+ 128us) server_negotiation.cc:407] Connection header received
0812 03:57:01.398278 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:01.398279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:01.398285 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:01.398371 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:01.398999 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:01.399000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:01.399001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:57:01.399005 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:01.399019 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:57:01.399024 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:01.399075 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:01.399151 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:57:01.399162 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:57:01.399789 10634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a36f8a02de433f901f84c53046dd17" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39325 } }
I20260812 03:57:01.399926 10634 leader_election.cc:302] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55a36f8a02de433f901f84c53046dd17; no voters:
I20260812 03:57:01.400102 10634 leader_election.cc:288] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:57:01.400152 10647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:01.400336 10647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [term 1 LEADER]: Becoming Leader. State: Replica: 55a36f8a02de433f901f84c53046dd17, State: Running, Role: LEADER
I20260812 03:57:01.400396 10634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:01.400425 10647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a36f8a02de433f901f84c53046dd17" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39325 } }
I20260812 03:57:01.400821 10648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55a36f8a02de433f901f84c53046dd17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a36f8a02de433f901f84c53046dd17" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39325 } } }
I20260812 03:57:01.400833 10649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a36f8a02de433f901f84c53046dd17. Latest consensus state: current_term: 1 leader_uuid: "55a36f8a02de433f901f84c53046dd17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a36f8a02de433f901f84c53046dd17" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 39325 } } }
I20260812 03:57:01.401048 10649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:01.401013 10648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:01.401362 10656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:01.402107 10656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:01.403726 10656 catalog_manager.cc:1347] Generated new cluster ID: 6daeab52126c43e2b12a37650595e849
I20260812 03:57:01.403784 10656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:01.413962 10656 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:01.414147 10656 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:01.424777 10656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55a36f8a02de433f901f84c53046dd17: Generated new TSK 0
I20260812 03:57:01.424989 10656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:57:01.433741 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 10556
2026-08-12T03:57:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1421 ms)
[----------] 18 tests from AuthTokenIssuingTest (25688 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:01 dist-test-slave-tm1g krb5kdc[10672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:01 dist-test-slave-tm1g krb5kdc[10672](info): set up 2 sockets
Aug 12 03:57:01 dist-test-slave-tm1g krb5kdc[10672](info): commencing operation
krb5kdc: starting...
W20260812 03:57:02.557637 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:57:02 dist-test-slave-tm1g krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:57:02.697896 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:35807
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:41369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:35807,127.17.54.253:45893,127.17.54.252:37065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:02.784896 10689 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:57:02.785145 10689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:02.785166 10689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:02.785188 10689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:02.786526 10689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:02.786576 10689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:02.786590 10689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:02.786603 10689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:02.786616 10689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:02.788256 10689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:35807,127.17.54.253:45893,127.17.54.252:37065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:35807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:02.788558 10689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:02.788846 10689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:02.794258 10697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:02.794368 10694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:02.794500 10689 server_base.cc:1034] running on GCE node
W20260812 03:57:02.794277 10695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:02.794868 10689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:02.795138 10689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:02.796322 10689 hybrid_clock.cc:648] HybridClock initialized: now 1786507022796316 us; error 57 us; skew 500 ppm
Aug 12 03:57:02 dist-test-slave-tm1g krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:02.799165 10689 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:57:02.799661 10689 webserver.cc:460] Webserver started at http://127.17.54.254:42221/ using document root <none> and password file <none>
I20260812 03:57:02.799917 10689 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:02.799991 10689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:02.800148 10689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:02.801278 10689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "cbb9653a33c44e428bee02b7736bc8ac"
format_stamp: "Formatted at 2026-08-12 03:57:02 on dist-test-slave-tm1g"
server_key: "7e2e0ca5ee8de7a236af3138f21edf59"
server_key_iv: "04065f188a5bb6b2709aef0143ad0f66"
server_key_version: "encryptionkey@0"
I20260812 03:57:02.801662 10689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "cbb9653a33c44e428bee02b7736bc8ac"
format_stamp: "Formatted at 2026-08-12 03:57:02 on dist-test-slave-tm1g"
server_key: "7e2e0ca5ee8de7a236af3138f21edf59"
server_key_iv: "04065f188a5bb6b2709aef0143ad0f66"
server_key_version: "encryptionkey@0"
I20260812 03:57:02.803234 10689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 03:57:02.804153 10705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:02.804432 10689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:02.804554 10689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "cbb9653a33c44e428bee02b7736bc8ac"
format_stamp: "Formatted at 2026-08-12 03:57:02 on dist-test-slave-tm1g"
server_key: "7e2e0ca5ee8de7a236af3138f21edf59"
server_key_iv: "04065f188a5bb6b2709aef0143ad0f66"
server_key_version: "encryptionkey@0"
I20260812 03:57:02.804637 10689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:02.830348 10689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:02.832899 10689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:02.833083 10689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:02.837503 10689 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:35807
I20260812 03:57:02.837646 10769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:35807 every 8 connection(s)
I20260812 03:57:02.837956 10689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:02.839040 10771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:02.840606 10771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:02.844691 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 10689
I20260812 03:57:02.844852 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:57:02.845047 17627 external_mini_cluster.cc:1428] Setting key 5404268fc4a7cd881c851b12d834f573
Aug 12 03:57:02 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:57:02.851609 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:02.841454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.17.54.254:35807)
0812 03:57:02.841654 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:02.841670 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:02.841888 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:57:02.842438 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:02.842441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:02.842458 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:02.842548 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:02.843131 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:02.843994 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:02.845077 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:02.845582 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:02.845752 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:02.848211 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:02.848218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:02.848218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:02.848224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:02.849869 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:02.850285 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:02.850288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:02.850288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:02.850413 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:02.850653 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:02.850653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:02.850654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:02.850892 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:02.850991 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:02.851190 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 03:57:02.851340 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
I20260812 03:57:02.851611 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:02.841368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:35807 (local address 127.0.0.1:38320)
0812 03:57:02.841625 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:02.841667 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:02.842209 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:02.842578 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:02.842580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:02.842838 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:02.843032 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:02.843036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:02.844111 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:02.844112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:02.844967 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:02.844970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:02.845735 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:02.845737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:02.845805 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:02.846419 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:02.846428 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:02.848101 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:02.849975 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:02.849982 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:02.849984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:02.850187 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:02.850484 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:02.850485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:02.850486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:02.850583 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:02.850968 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:02.850973 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:02.851043 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 03:57:02.851154 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:57:02.851406 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":98,"thread_start_us":133,"threads_started":1}
I20260812 03:57:02.852443 10771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:02.852944 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:02.852714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49810)
0812 03:57:02.852868 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:02.852918 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260812 03:57:02.853163 10710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.54.253@KRBTEST.COM; defaulting to no policy
I20260812 03:57:02.853767 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:02.853572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49824)
0812 03:57:02.853692 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:02.853737 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260812 03:57:02.853902 10771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:45893: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab.
Entry for principal kudu/127.17.54.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab.
I20260812 03:57:02.877192 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.253:45893
--webserver_interface=127.17.54.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab
--principal=kudu/127.17.54.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:41369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:35807,127.17.54.253:45893,127.17.54.252:37065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:57:02.903137 10771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } attempt: 1
I20260812 03:57:02.903807 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:02.903666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49838)
0812 03:57:02.903734 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:02.903775 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:57:02.904179 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:02.904074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49850)
0812 03:57:02.904124 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:02.904158 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:57:02.904266 10771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:45893: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
W20260812 03:57:02.967455 10779 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:57:02.967751 10779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:02.967788 10779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:02.967840 10779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:02.969169 10779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:02.969254 10779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:02.969285 10779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:02.969321 10779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:02.969347 10779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:02.971179 10779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:35807,127.17.54.253:45893,127.17.54.252:37065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab
--principal=kudu/127.17.54.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.253:45893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.54.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:02.971501 10779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:02.971767 10779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:57:02.972443 10771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } attempt: 2
I20260812 03:57:02.972926 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:02.972780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49862)
0812 03:57:02.972853 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:02.972895 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 03:57:02.973318 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:02.973202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49868)
0812 03:57:02.973244 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:02.973270 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 03:57:02.973495 10771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:45893: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45893: connect: Connection refused (error 111)
W20260812 03:57:02.977352 10784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:02.977411 10787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:02.977427 10785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:02.977874 10779 server_base.cc:1034] running on GCE node
I20260812 03:57:02.978185 10779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:02.978506 10779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:02.979698 10779 hybrid_clock.cc:648] HybridClock initialized: now 1786507022979682 us; error 25 us; skew 500 ppm
Aug 12 03:57:02 dist-test-slave-tm1g krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:02.982422 10779 init.cc:377] Logged in from keytab as kudu/127.17.54.253@KRBTEST.COM (short username kudu)
I20260812 03:57:02.982873 10779 webserver.cc:460] Webserver started at http://127.17.54.253:39385/ using document root <none> and password file <none>
I20260812 03:57:02.983131 10779 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:02.983203 10779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:02.983353 10779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:02.984467 10779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/data/instance:
uuid: "c131bc97b4d34d10a0029c18b4f17b62"
format_stamp: "Formatted at 2026-08-12 03:57:02 on dist-test-slave-tm1g"
server_key: "9838abd26abd1e34feba9d69ee98cd1c"
server_key_iv: "f20ed2a24dd291165f1a63ba7ed73963"
server_key_version: "encryptionkey@0"
I20260812 03:57:02.984831 10779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/wal/instance:
uuid: "c131bc97b4d34d10a0029c18b4f17b62"
format_stamp: "Formatted at 2026-08-12 03:57:02 on dist-test-slave-tm1g"
server_key: "9838abd26abd1e34feba9d69ee98cd1c"
server_key_iv: "f20ed2a24dd291165f1a63ba7ed73963"
server_key_version: "encryptionkey@0"
I20260812 03:57:02.986234 10779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:02.987136 10796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:02.987380 10779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:02.987506 10779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/wal
uuid: "c131bc97b4d34d10a0029c18b4f17b62"
format_stamp: "Formatted at 2026-08-12 03:57:02 on dist-test-slave-tm1g"
server_key: "9838abd26abd1e34feba9d69ee98cd1c"
server_key_iv: "f20ed2a24dd291165f1a63ba7ed73963"
server_key_version: "encryptionkey@0"
I20260812 03:57:02.987584 10779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:03.013626 10779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:03.016346 10779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:03.016534 10779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:03.021174 10779 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.253:45893
I20260812 03:57:03.021234 10858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.253:45893 every 8 connection(s)
I20260812 03:57:03.021610 10779 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
I20260812 03:57:03.022804 10859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:03.023947 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 10779
I20260812 03:57:03.024101 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-1/wal/instance
I20260812 03:57:03.024315 17627 external_mini_cluster.cc:1428] Setting key b21281f84097341ed490b743c4b2e736
I20260812 03:57:03.024592 10859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.54.252@KRBTEST.COM; defaulting to no policy
I20260812 03:57:03.036113 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.025369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38334 (local address 127.17.54.254:35807)
0812 03:57:03.025455 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.025458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.025752 (+ 294us) server_negotiation.cc:407] Connection header received
0812 03:57:03.026646 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.026647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.026661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.026723 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.027442 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.028301 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.029334 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.029750 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.029846 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.032706 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.032708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.032709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.032718 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.034532 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.035005 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.035007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.035007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.035082 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.035361 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.035363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.035363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.035500 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.035579 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.035905 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.036012 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:57:03.036350 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.025339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:35807 (local address 127.0.0.1:38334)
0812 03:57:03.025589 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.025649 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:03.026516 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:03.026796 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:03.026800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:03.027101 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:03.027322 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.027327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.028458 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.028460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.029257 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.029259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.029904 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.029905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.030062 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.030732 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:03.030744 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:03.032583 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:03.034623 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.034632 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.034634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.034890 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.035160 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.035161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.035161 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.035287 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.035581 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:03.035590 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:03.035753 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 03:57:03.035867 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:57:03.036045 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.252@KRBTEST.COM" created.
I20260812 03:57:03.037163 10859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:57:03.045744 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.037456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49878)
0812 03:57:03.037707 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.037720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:03.037765 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:03.037927 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:03.037928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:03.038109 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:03.038181 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.038184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.039327 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.039328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.039848 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.039850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.040346 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.040347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.040414 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.040887 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:03.040892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:03.042142 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:03.044400 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.044400 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.044401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.044601 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.044866 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.044868 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.044868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.044935 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.045305 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:03.045306 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:03.045356 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:57:03.045430 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:57:03.045588 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260812 03:57:03.045825 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.037541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49878 (local address 127.17.54.253:45893)
0812 03:57:03.037733 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.037737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.037760 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:57:03.037821 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.037822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.037834 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.037905 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.038259 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.039193 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.039947 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.040268 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.040351 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.042242 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.042253 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.042254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.042268 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.044247 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.044715 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.044718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.044719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.044786 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.045009 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.045010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.045010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.045188 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.045374 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.045463 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.045668 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
I20260812 03:57:03.046557 10859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:03.047019 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.046879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57828)
0812 03:57:03.046935 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.046993 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:57:03.047224 10799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
I20260812 03:57:03.047755 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.047611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57836)
0812 03:57:03.047686 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.047726 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 03:57:03.047899 10859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37065: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab.
Entry for principal kudu/127.17.54.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab.
I20260812 03:57:03.054563 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.252:37065
--webserver_interface=127.17.54.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab
--principal=kudu/127.17.54.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:41369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:35807,127.17.54.253:45893,127.17.54.252:37065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:57:03.064677 10771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } attempt: 3
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:57:03.072405 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.065348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49882 (local address 127.17.54.253:45893)
0812 03:57:03.065420 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.065422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.065432 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:03.065472 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.065473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.065484 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.065578 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.065941 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.066531 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.067277 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.067515 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.067596 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.069686 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.069687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.069687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.069698 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.071213 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.071598 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.071599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.071599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.071660 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.071865 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.071866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.071866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.071987 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.072064 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.072226 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.072303 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:57:03.072408 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.065149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49882)
0812 03:57:03.065245 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.065260 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:03.065354 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:03.065585 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:03.065586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:03.065764 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:03.065851 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.065853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.066615 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.066617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.067185 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.067188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.067594 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.067595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.067676 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.068162 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:03.068167 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:03.069586 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:03.071326 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.071327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.071327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.071504 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.071737 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.071738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.071738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.071800 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.072056 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:03.072057 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:03.072105 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:57:03.072185 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:57:03.072298 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 03:57:03.073148 10771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:03.073585 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.073424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57844)
0812 03:57:03.073516 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.073552 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 03:57:03.074040 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.073917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57858)
0812 03:57:03.073981 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.074008 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:57:03.074137 10771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37065: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
I20260812 03:57:03.097210 10859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } attempt: 1
I20260812 03:57:03.097796 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.097638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57868)
0812 03:57:03.097709 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.097759 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 03:57:03.098248 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.098149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57880)
0812 03:57:03.098198 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.098227 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 03:57:03.098392 10859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37065: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
I20260812 03:57:03.105302 10771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } attempt: 1
I20260812 03:57:03.105829 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.105690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57884)
0812 03:57:03.105754 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.105790 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:57:03.106233 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.106139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57888)
0812 03:57:03.106187 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.106215 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 03:57:03.106364 10771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37065: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
W20260812 03:57:03.141129 10866 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:57:03.141412 10866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:03.141445 10866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:03.141490 10866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:03.142699 10866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:03.142783 10866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:03.142808 10866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:03.142835 10866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:03.142868 10866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:03.144408 10866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:41369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:35807,127.17.54.253:45893,127.17.54.252:37065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab
--principal=kudu/127.17.54.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.252:37065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.54.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:03.144709 10866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:03.144963 10866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:57:03.150315 10866 server_base.cc:1034] running on GCE node
W20260812 03:57:03.150238 10874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:03.150206 10872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:03.150211 10871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:03.150864 10866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:03.151221 10866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:03.152372 10866 hybrid_clock.cc:648] HybridClock initialized: now 1786507023152341 us; error 55 us; skew 500 ppm
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507023, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:03.155211 10866 init.cc:377] Logged in from keytab as kudu/127.17.54.252@KRBTEST.COM (short username kudu)
I20260812 03:57:03.155675 10866 webserver.cc:460] Webserver started at http://127.17.54.252:42661/ using document root <none> and password file <none>
I20260812 03:57:03.156002 10866 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:03.156049 10866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:03.156183 10866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:03.157349 10866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/data/instance:
uuid: "ccde009710184793b4d0757aeed12332"
format_stamp: "Formatted at 2026-08-12 03:57:03 on dist-test-slave-tm1g"
server_key: "3d6766ec76cfea9dc4653ad62fbbb1a3"
server_key_iv: "994f0cb47f6c4fbfaaf185aa9a4655e7"
server_key_version: "encryptionkey@0"
I20260812 03:57:03.157723 10866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/wal/instance:
uuid: "ccde009710184793b4d0757aeed12332"
format_stamp: "Formatted at 2026-08-12 03:57:03 on dist-test-slave-tm1g"
server_key: "3d6766ec76cfea9dc4653ad62fbbb1a3"
server_key_iv: "994f0cb47f6c4fbfaaf185aa9a4655e7"
server_key_version: "encryptionkey@0"
I20260812 03:57:03.159206 10866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:03.160040 10882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:03.160218 10866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:03.160318 10866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/wal
uuid: "ccde009710184793b4d0757aeed12332"
format_stamp: "Formatted at 2026-08-12 03:57:03 on dist-test-slave-tm1g"
server_key: "3d6766ec76cfea9dc4653ad62fbbb1a3"
server_key_iv: "994f0cb47f6c4fbfaaf185aa9a4655e7"
server_key_version: "encryptionkey@0"
I20260812 03:57:03.160420 10866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:03.166548 10859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } attempt: 2
I20260812 03:57:03.166996 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.166852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57892)
0812 03:57:03.166915 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.166956 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:57:03.167351 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.167246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57906)
0812 03:57:03.167297 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.167330 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:57:03.167435 10859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37065: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
I20260812 03:57:03.181525 10771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } attempt: 2
I20260812 03:57:03.182091 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.181934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57912)
0812 03:57:03.182000 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.182048 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:57:03.182726 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.182636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57916)
0812 03:57:03.182682 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.182709 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:57:03.182830 10771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:37065: Network error: Client connection negotiation failed: client connection to 127.17.54.252:37065: connect: Connection refused (error 111)
I20260812 03:57:03.192657 10866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:03.195039 10866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:03.195155 10866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:03.199308 10866 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.252:37065
I20260812 03:57:03.199414 10945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.252:37065 every 8 connection(s)
I20260812 03:57:03.199752 10866 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
I20260812 03:57:03.200814 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 10866
I20260812 03:57:03.200917 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506741478267-17627-0/minicluster-data/master-2/wal/instance
I20260812 03:57:03.201102 17627 external_mini_cluster.cc:1428] Setting key 174d4cc65ce5c0b7ee4f10fc05919b89
I20260812 03:57:03.201094 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:03.203776 10946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507023, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:57:03.212806 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.201728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57918 (local address 127.17.54.252:37065)
0812 03:57:03.201910 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.201919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.201955 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:57:03.202412 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.202419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.202476 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.202589 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.203439 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.204791 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.205762 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.206298 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.206434 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.209090 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.209098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.209098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.209105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.210947 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.211329 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.211332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.211333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.211425 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.211654 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.211655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.211655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.211968 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.212127 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.212366 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.212558 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260812 03:57:03.213080 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.204507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38338 (local address 127.17.54.254:35807)
0812 03:57:03.204574 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.204576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.204742 (+ 166us) server_negotiation.cc:407] Connection header received
0812 03:57:03.204825 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.204826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.204866 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.204926 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.205432 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.205993 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.207068 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.207435 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.207524 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.210207 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.210209 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.210209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.210215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.211716 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.212157 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.212158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.212158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.212229 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.212451 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.212452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.212452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.212594 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.212675 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.212902 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.212987 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:03.213095 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.204452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:35807 (local address 127.0.0.1:38338)
0812 03:57:03.204635 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.204675 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:03.204760 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:03.204970 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:03.204972 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:03.205246 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:03.205340 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.205342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.206094 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.206095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.206991 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.206994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.207582 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.207584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.207648 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.208202 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:03.208211 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:03.209909 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:03.211792 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.211797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.211800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.212074 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.212302 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.212303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.212303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.212369 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.212680 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:03.212688 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:03.212740 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:57:03.212867 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 03:57:03.212988 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260812 03:57:03.213729 10946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507023, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:57:03.221568 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.214046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.17.54.253:45893)
0812 03:57:03.214090 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.214091 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.214102 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:03.214128 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.214129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.214144 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.214186 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.214490 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.215234 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.216223 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.216528 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.216610 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.218661 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.218663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.218663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.218670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.220222 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.220705 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.220707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.220707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.220775 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.220967 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.220968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.220968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.221129 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.221206 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.221427 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.221502 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:57:03.221619 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.213985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45893 (local address 127.0.0.1:49888)
0812 03:57:03.214036 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.214046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:03.214082 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:03.214194 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:03.214194 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:03.214367 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:03.214420 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.214422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.215325 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.215326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.216111 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.216113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.216629 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.216630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.216694 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.217111 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:03.217116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:03.218569 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:03.220356 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.220357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.220358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.220631 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.220846 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.220846 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.220846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.220902 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.221224 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:03.221225 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:03.221267 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:57:03.221391 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 03:57:03.221491 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:57:03.222361 10946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:03.222606 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.213706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.17.54.254:35807)
0812 03:57:03.213738 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.213740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.213766 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:57:03.213784 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.213787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.213799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.213867 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.214494 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.215235 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.216220 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.216555 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.216615 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.219514 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.219515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.219515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.219522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.221292 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.221700 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.221701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.221701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.221765 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.222014 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.222016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.222016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.222183 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.222392 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.222435 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.222515 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507023, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:57:03.229509 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.222717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57934)
0812 03:57:03.222795 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.222805 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:03.222845 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:03.223007 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:03.223008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:03.223171 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:03.223206 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.223208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.223838 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.223839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.224303 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.224304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.224677 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.224678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.224726 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.225128 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:03.225132 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:03.226634 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:03.228256 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.228257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.228258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.228503 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.228731 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.228732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.228733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.228786 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.229154 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:03.229155 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:03.229207 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:57:03.229285 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:57:03.229372 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 03:57:03.229509 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.222751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.17.54.252:37065)
0812 03:57:03.222907 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.222908 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.222913 (+ 5us) server_negotiation.cc:407] Connection header received
0812 03:57:03.222930 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.222930 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.222943 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.223003 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.223281 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.223752 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.224381 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.224594 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.224678 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.226734 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.226735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.226735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.226745 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.228168 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.228580 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.228581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.228581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.228632 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.228978 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.228979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.228979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.229082 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.229136 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.229372 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.229442 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260812 03:57:03.231118 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.223209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.17.54.253:45893)
0812 03:57:03.223255 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.223257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.223267 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:03.223288 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.223289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.223297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.223352 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.223805 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.224253 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.224903 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.225129 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.225206 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.227889 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.227890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.227891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.227897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.229703 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.230296 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.230297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.230297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.230421 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.230626 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.230627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.230627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.230731 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.230841 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.230947 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.231032 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:03.231421 10946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332: Bootstrap starting.
I20260812 03:57:03.232493 10946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:03.232642 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:57:03.232841 10946 log.cc:824] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:03.233672 10946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332: No bootstrap required, opened a new log
I20260812 03:57:03.233747 10736 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cbb9653a33c44e428bee02b7736bc8ac"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38350
I20260812 03:57:03.235230 10946 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } }
I20260812 03:57:03.235356 10946 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccde009710184793b4d0757aeed12332, State: Initialized, Role: FOLLOWER
I20260812 03:57:03.235447 10946 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } }
I20260812 03:57:03.235764 10952 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.235777 10946 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:03.235888 10952 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:57:03.237689 10965 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:03.237778 10965 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:03.238054 10866 master_runner.cc:186] Error getting master registration for 127.17.54.254:35807: OK, instance_id { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" instance_seqno: 1786507022836242 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:57:03.241036 10866 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:57:03.258589 10859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } attempt: 3
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
I20260812 03:57:03.268038 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.259040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57936)
0812 03:57:03.259104 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.259117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:03.259171 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:03.259336 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:03.259337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:03.259559 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:03.259620 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.259622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.260598 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.260600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.262032 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.262036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.262592 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.262593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.262656 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.263186 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:03.263191 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:03.264794 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:03.266735 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.266737 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.266737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.267017 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.267283 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.267283 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.267284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.267350 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.267687 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:03.267688 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:03.267736 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:57:03.267806 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:57:03.267919 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 03:57:03.268038 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.259142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.17.54.252:37065)
0812 03:57:03.259190 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.259193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.259216 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:57:03.259245 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.259245 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.259262 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.259327 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.259716 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.260499 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.262147 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.262508 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.262588 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.264888 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.264890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.264890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.264900 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.266618 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.267097 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.267098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.267098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.267163 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.267413 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.267413 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.267414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.267612 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.267686 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.267849 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.267934 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:03.269788 10859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62: Bootstrap starting.
I20260812 03:57:03.270527 10859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:03.270838 10859 log.cc:824] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:03.271608 10859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62: No bootstrap required, opened a new log
I20260812 03:57:03.273046 10859 raft_consensus.cc:348] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } }
I20260812 03:57:03.273195 10859 raft_consensus.cc:729] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c131bc97b4d34d10a0029c18b4f17b62, State: Initialized, Role: FOLLOWER
I20260812 03:57:03.273295 10859 consensus_queue.cc:260] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } }
I20260812 03:57:03.273648 10969 sys_catalog.cc:455] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.273725 10859 sys_catalog.cc:564] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:03.273780 10969 sys_catalog.cc:458] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:57:03.275664 10981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:03.275751 10981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:03.276063 10779 master_runner.cc:186] Error getting master registration for 127.17.54.254:35807: OK, instance_id { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" instance_seqno: 1786507022836242 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:57:03.278599 10779 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:57:03.281988 10771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } attempt: 3
Aug 12 03:57:03 dist-test-slave-tm1g krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507022, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
I20260812 03:57:03.293375 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.282290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:37065 (local address 127.0.0.1:57950)
0812 03:57:03.282439 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:03.282452 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:03.282497 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:03.284840 (+ 2343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:03.284842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:03.285046 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:03.285105 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.285107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.286603 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.286604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.287434 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:03.287437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.287948 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.287949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:03.288013 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.288522 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:03.288528 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:03.290167 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:03.292133 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.292134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.292134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.292396 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.292631 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:03.292632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:03.292632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:03.292698 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:03.293006 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:03.293007 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:03.293076 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 03:57:03.293142 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:57:03.293250 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260812 03:57:03.293907 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.282573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.17.54.252:37065)
0812 03:57:03.282612 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.282614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.282621 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:57:03.282647 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.282648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.282658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.284762 (+ 2104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.285197 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.286498 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.287528 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.287845 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.287936 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.290346 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.290348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.290348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.290356 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.292045 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.292478 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.292479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.292480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.292555 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.292757 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.292759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.292759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.292933 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.293011 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.293725 (+ 714us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.293813 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:57:03.295697 10771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac: Bootstrap starting.
I20260812 03:57:03.296484 10771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:03.296813 10771 log.cc:824] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:03.297731 10771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac: No bootstrap required, opened a new log
I20260812 03:57:03.299180 10771 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } }
I20260812 03:57:03.299350 10771 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbb9653a33c44e428bee02b7736bc8ac, State: Initialized, Role: FOLLOWER
I20260812 03:57:03.299443 10771 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } }
I20260812 03:57:03.299782 10771 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:03.300032 10736 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:57:03.300110 10736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:03.300976 10736 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } }
I20260812 03:57:03.301268 10736 leader_election.cc:288] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [CANDIDATE]: Term 1 election: Requested vote from peers c131bc97b4d34d10a0029c18b4f17b62 (127.17.54.253:45893), ccde009710184793b4d0757aeed12332 (127.17.54.252:37065)
I20260812 03:57:03.301662 10829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbb9653a33c44e428bee02b7736bc8ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c131bc97b4d34d10a0029c18b4f17b62"
I20260812 03:57:03.301726 10916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbb9653a33c44e428bee02b7736bc8ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ccde009710184793b4d0757aeed12332"
I20260812 03:57:03.301806 10829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:03.301862 10916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:03.302889 10829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbb9653a33c44e428bee02b7736bc8ac in term 1.
I20260812 03:57:03.302935 10916 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbb9653a33c44e428bee02b7736bc8ac in term 1.
I20260812 03:57:03.304132 10710 leader_election.cc:302] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c131bc97b4d34d10a0029c18b4f17b62, cbb9653a33c44e428bee02b7736bc8ac; no voters:
I20260812 03:57:03.304436 10984 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.304538 10984 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:03.304960 10984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:03.305078 10984 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [term 1 LEADER]: Becoming Leader. State: Replica: cbb9653a33c44e428bee02b7736bc8ac, State: Running, Role: LEADER
I20260812 03:57:03.305157 10984 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } }
I20260812 03:57:03.305613 10984 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb9653a33c44e428bee02b7736bc8ac. Latest consensus state: current_term: 1 leader_uuid: "cbb9653a33c44e428bee02b7736bc8ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.305680 10984 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: This master's current role is: LEADER
W20260812 03:57:03.307214 10995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:03.307264 10995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:03.307572 10689 master_runner.cc:263] No leader master found from master cbb9653a33c44e428bee02b7736bc8ac
I20260812 03:57:03.307638 10689 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:57:03.308322 11004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:03.309886 11004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:03.317921 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.305985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49906 (local address 127.17.54.253:45893)
0812 03:57:03.306026 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.306028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.309828 (+ 3800us) server_negotiation.cc:407] Connection header received
0812 03:57:03.309925 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.309926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.309937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.309966 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.310800 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.311553 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.312599 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.312871 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.312917 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.314622 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.314624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.314624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.314632 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.316530 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.317003 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.317004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.317005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.317068 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.317290 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.317291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.317291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.317437 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.317524 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.317740 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.317825 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:57:03.320422 10916 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [term 1 FOLLOWER]: Refusing update from remote peer cbb9653a33c44e428bee02b7736bc8ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:03.320815 11002 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:03.322381 10952 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb9653a33c44e428bee02b7736bc8ac. Latest consensus state: current_term: 1 leader_uuid: "cbb9653a33c44e428bee02b7736bc8ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.322504 10952 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:03.322932 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.304554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.17.54.254:35807)
0812 03:57:03.304603 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.304606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.308585 (+ 3979us) server_negotiation.cc:407] Connection header received
0812 03:57:03.308848 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.308849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.308860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.308895 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.310855 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.311686 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.315420 (+ 3734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.315797 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.315849 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.317528 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.317530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.317530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.317539 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.319777 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.320386 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.320388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.320388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.320461 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.321115 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.321116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.321117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.321275 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.321385 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.322725 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.322828 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:57:03.322944 10829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [term 1 FOLLOWER]: Refusing update from remote peer cbb9653a33c44e428bee02b7736bc8ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:03.323226 11002 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:03.324952 10969 sys_catalog.cc:455] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb9653a33c44e428bee02b7736bc8ac. Latest consensus state: current_term: 1 leader_uuid: "cbb9653a33c44e428bee02b7736bc8ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.325054 10969 sys_catalog.cc:458] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:03.325847 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:03.306461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.17.54.252:37065)
0812 03:57:03.306499 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.306501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.318475 (+ 11974us) server_negotiation.cc:407] Connection header received
0812 03:57:03.318602 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.318603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.318616 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.318651 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.319410 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.320112 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.321116 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:03.321421 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:03.321460 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:03.322914 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:03.322915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:03.322916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:03.322924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:03.324565 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.324963 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.324964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.324965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.325023 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:03.325224 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:03.325224 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:03.325225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:03.325387 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:03.325457 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:03.325676 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 03:57:03.325758 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:57:03.331554 10984 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cbb9653a33c44e428bee02b7736bc8ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.331858 10969 sys_catalog.cc:455] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cbb9653a33c44e428bee02b7736bc8ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.331964 10969 sys_catalog.cc:458] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:03.332167 11013 mvcc.cc:205] Tried to move back new op lower bound from 7317532767517753344 to 7317532767458783232. Current Snapshot: MvccSnapshot[applied={T|T < 7317532767517753344}]
I20260812 03:57:03.343484 10984 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:03.343642 11010 mvcc.cc:205] Tried to move back new op lower bound from 7317532767517753344 to 7317532767458783232. Current Snapshot: MvccSnapshot[applied={T|T < 7317532767517753344}]
I20260812 03:57:03.344213 10984 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cbb9653a33c44e428bee02b7736bc8ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.344319 10984 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:03.344844 11004 catalog_manager.cc:1347] Generated new cluster ID: e2ece919ef4b46b2abe6adb492ca8153
I20260812 03:57:03.344900 11004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:03.344888 10952 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cbb9653a33c44e428bee02b7736bc8ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb9653a33c44e428bee02b7736bc8ac" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 35807 } } peers { permanent_uuid: "c131bc97b4d34d10a0029c18b4f17b62" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45893 } } peers { permanent_uuid: "ccde009710184793b4d0757aeed12332" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 37065 } } }
I20260812 03:57:03.344972 10952 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:03.345316 11011 mvcc.cc:205] Tried to move back new op lower bound from 7317532767517753344 to 7317532767458783232. Current Snapshot: MvccSnapshot[applied={T|T < 7317532767517753344}]
I20260812 03:57:03.356210 11004 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:03.357028 11004 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:03.370909 11004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbb9653a33c44e428bee02b7736bc8ac: Generated new TSK 0
I20260812 03:57:03.371132 11004 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:57:03.400658 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.398635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.17.54.253:45893)
0812 03:57:03.398689 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.398692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.399867 (+ 1175us) server_negotiation.cc:407] Connection header received
0812 03:57:03.400005 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.400007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.400020 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.400053 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.400576 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49908: BlockingRecv error: recv got EOF from 127.0.0.1:49908 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260812 03:57:03.401718 10949 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.399153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.17.54.252:37065)
0812 03:57:03.399201 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.399204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.401129 (+ 1925us) server_negotiation.cc:407] Connection header received
0812 03:57:03.401237 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.401238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.401250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.401283 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.401651 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57966: BlockingRecv error: recv got EOF from 127.0.0.1:57966 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 03:57:03.402719 10775 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.399533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.17.54.254:35807)
0812 03:57:03.399603 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.399605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.402109 (+ 2504us) server_negotiation.cc:407] Connection header received
0812 03:57:03.402214 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.402215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.402227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.402260 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.402652 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38368: BlockingRecv error: recv got EOF from 127.0.0.1:38368 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 03:57:03.405407 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.404240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.17.54.253:45893)
0812 03:57:03.404282 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.404283 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.404790 (+ 507us) server_negotiation.cc:407] Connection header received
0812 03:57:03.404924 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.404925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.404946 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.404981 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.405353 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49912: BlockingRecv error: recv got EOF from 127.0.0.1:49912 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 03:57:03.408918 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.407930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49914 (local address 127.17.54.253:45893)
0812 03:57:03.407969 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.407971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.408266 (+ 295us) server_negotiation.cc:407] Connection header received
0812 03:57:03.408376 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.408377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.408389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.408423 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.408851 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49914: BlockingRecv error: recv got EOF from 127.0.0.1:49914 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 03:57:03.413338 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.412396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49918 (local address 127.17.54.253:45893)
0812 03:57:03.412438 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.412440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.412721 (+ 281us) server_negotiation.cc:407] Connection header received
0812 03:57:03.412846 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.412848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.412860 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.412910 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.413283 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49918: BlockingRecv error: recv got EOF from 127.0.0.1:49918 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 03:57:03.419584 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.418828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49924 (local address 127.17.54.253:45893)
0812 03:57:03.418871 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.418873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.419046 (+ 173us) server_negotiation.cc:407] Connection header received
0812 03:57:03.419079 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.419080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.419094 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.419157 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.419526 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49924: BlockingRecv error: recv got EOF from 127.0.0.1:49924 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 03:57:03.429608 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.428738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.17.54.253:45893)
0812 03:57:03.428786 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.428787 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.428923 (+ 136us) server_negotiation.cc:407] Connection header received
0812 03:57:03.429043 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.429044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.429055 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.429129 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.429534 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49930: BlockingRecv error: recv got EOF from 127.0.0.1:49930 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 03:57:03.447816 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.446939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.17.54.253:45893)
0812 03:57:03.446991 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.446994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.447140 (+ 146us) server_negotiation.cc:407] Connection header received
0812 03:57:03.447201 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.447203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.447216 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.447299 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.447752 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49934: BlockingRecv error: recv got EOF from 127.0.0.1:49934 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 03:57:03.481940 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.481124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.17.54.253:45893)
0812 03:57:03.481176 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.481179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.481312 (+ 133us) server_negotiation.cc:407] Connection header received
0812 03:57:03.481411 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.481413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.481427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.481487 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.481856 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49936: BlockingRecv error: recv got EOF from 127.0.0.1:49936 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 03:57:03.548395 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.547573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.17.54.253:45893)
0812 03:57:03.547679 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.547681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.547774 (+ 93us) server_negotiation.cc:407] Connection header received
0812 03:57:03.547854 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.547856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.547875 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.547938 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.548312 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49944: BlockingRecv error: recv got EOF from 127.0.0.1:49944 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 03:57:03.678790 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.677958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.17.54.253:45893)
0812 03:57:03.678043 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.678046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.678097 (+ 51us) server_negotiation.cc:407] Connection header received
0812 03:57:03.678159 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.678160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.678171 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.678238 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.678738 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49946: BlockingRecv error: recv got EOF from 127.0.0.1:49946 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 03:57:03.937209 10863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:03.936272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.17.54.253:45893)
0812 03:57:03.936347 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:03.936393 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:03.936407 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:57:03.936589 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:03.936591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:03.936603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:03.936659 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:03.937137 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49954: BlockingRecv error: recv got EOF from 127.0.0.1:49954 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:57:04.238327 10965 catalog_manager.cc:1259] Loaded cluster ID: e2ece919ef4b46b2abe6adb492ca8153
I20260812 03:57:04.238410 10965 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332: loading cluster ID for follower catalog manager: success
I20260812 03:57:04.239313 10965 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332: acquiring CA information for follower catalog manager: success
I20260812 03:57:04.239630 10965 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ccde009710184793b4d0757aeed12332: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:57:04.276257 10981 catalog_manager.cc:1259] Loaded cluster ID: e2ece919ef4b46b2abe6adb492ca8153
I20260812 03:57:04.276337 10981 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62: loading cluster ID for follower catalog manager: success
I20260812 03:57:04.277117 10981 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62: acquiring CA information for follower catalog manager: success
I20260812 03:57:04.277367 10981 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c131bc97b4d34d10a0029c18b4f17b62: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:57:04.453106 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:04.450362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49966 (local address 127.17.54.253:45893)
0812 03:57:04.450541 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:04.450543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:04.450557 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:57:04.450630 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:04.450630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:04.450683 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:04.450762 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:04.450905 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.451436 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.452203 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.452401 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.452465 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:04.452666 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:04.452778 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:04.452838 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:04.452952 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:57:04.452972 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260812 03:57:04.458163 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:04.455520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57982 (local address 127.17.54.252:37065)
0812 03:57:04.455653 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:04.455655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:04.455664 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:57:04.455711 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:04.455711 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:04.455800 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:04.455905 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:04.456042 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.456537 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.457332 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.457541 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.457618 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:04.457791 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:04.457904 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:04.457996 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:04.458037 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 03:57:04.458059 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260812 03:57:04.462688 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:04.459951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57996 (local address 127.17.54.252:37065)
0812 03:57:04.459996 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:04.459999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:04.460125 (+ 126us) server_negotiation.cc:407] Connection header received
0812 03:57:04.460190 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:04.460191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:04.460203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:04.460261 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:04.460421 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.461143 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.461958 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.462188 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.462251 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:04.462429 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:04.462460 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:04.462519 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:04.462575 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 03:57:04.462595 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:04.463038 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:04.459851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49970 (local address 127.17.54.253:45893)
0812 03:57:04.459902 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:04.459905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:04.460275 (+ 370us) server_negotiation.cc:407] Connection header received
0812 03:57:04.460362 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:04.460363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:04.460375 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:04.460420 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:04.460591 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.461064 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.462135 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.462471 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.462533 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:04.462717 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:04.462752 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:04.462783 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:04.462927 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:57:04.462955 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:57:04.463529 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:04.459937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38378 (local address 127.17.54.254:35807)
0812 03:57:04.460172 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:04.460174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:04.460180 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:57:04.460216 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:04.460216 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:04.460287 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:04.460414 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:04.460526 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.461261 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.462355 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:04.462749 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:04.462962 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:04.463040 (+ 78us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:04.463178 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:04.463409 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:04.463428 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 03:57:04.463453 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":61,"threads_started":1}
I20260812 03:57:04.465196 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 10689
I20260812 03:57:04.473840 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 10779
I20260812 03:57:04.482291 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 10866
2026-08-12T03:57:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3048 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:57:04 dist-test-slave-tm1g krb5kdc[11115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:57:04 dist-test-slave-tm1g krb5kdc[11115](info): set up 2 sockets
Aug 12 03:57:04 dist-test-slave-tm1g krb5kdc[11115](info): commencing operation
krb5kdc: starting...
W20260812 03:57:05.601182 17627 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/test-user.keytab.
Aug 12 03:57:05 dist-test-slave-tm1g krb5kdc[11115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:57:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:57:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.54.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
Entry for principal kudu/127.17.54.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab.
I20260812 03:57:05.729482 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37203
--webserver_interface=127.17.54.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:37203,127.17.54.253:45497,127.17.54.252:40303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:57:05.818974 11130 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:57:05.819227 11130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:05.819284 11130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:05.819327 11130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:05.820514 11130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:05.820590 11130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:05.820616 11130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:05.820647 11130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:05.820674 11130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:05.822382 11130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:37203,127.17.54.253:45497,127.17.54.252:40303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.254.keytab
--principal=kudu/127.17.54.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.254:37203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.54.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:05.822666 11130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:05.822921 11130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:05.828207 11138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:05.828359 11130 server_base.cc:1034] running on GCE node
W20260812 03:57:05.828210 11136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:05.828450 11135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:05.828692 11130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:05.829027 11130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:05.830214 11130 hybrid_clock.cc:648] HybridClock initialized: now 1786507025830195 us; error 46 us; skew 500 ppm
Aug 12 03:57:05 dist-test-slave-tm1g krb5kdc[11115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507025, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:05.841178 11130 init.cc:377] Logged in from keytab as kudu/127.17.54.254@KRBTEST.COM (short username kudu)
I20260812 03:57:05.841742 11130 webserver.cc:460] Webserver started at http://127.17.54.254:33493/ using document root <none> and password file <none>
I20260812 03:57:05.842025 11130 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:05.842099 11130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:05.842257 11130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:05.843427 11130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/data/instance:
uuid: "5773951dc9c54dadafbbb1eb1766a6a0"
format_stamp: "Formatted at 2026-08-12 03:57:05 on dist-test-slave-tm1g"
server_key: "fcfa039a39c257bc0e01444c26253770"
server_key_iv: "6029d0a88b3a0f7a7204c96d1db85f69"
server_key_version: "encryptionkey@0"
I20260812 03:57:05.843809 11130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/wal/instance:
uuid: "5773951dc9c54dadafbbb1eb1766a6a0"
format_stamp: "Formatted at 2026-08-12 03:57:05 on dist-test-slave-tm1g"
server_key: "fcfa039a39c257bc0e01444c26253770"
server_key_iv: "6029d0a88b3a0f7a7204c96d1db85f69"
server_key_version: "encryptionkey@0"
I20260812 03:57:05.845225 11130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:05.846079 11145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:05.846290 11130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:05.846447 11130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/wal
uuid: "5773951dc9c54dadafbbb1eb1766a6a0"
format_stamp: "Formatted at 2026-08-12 03:57:05 on dist-test-slave-tm1g"
server_key: "fcfa039a39c257bc0e01444c26253770"
server_key_iv: "6029d0a88b3a0f7a7204c96d1db85f69"
server_key_version: "encryptionkey@0"
I20260812 03:57:05.846530 11130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:05.870131 11130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:05.872617 11130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:05.872800 11130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:05.877525 11130 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.254:37203
I20260812 03:57:05.877573 11205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.254:37203 every 8 connection(s)
I20260812 03:57:05.877967 11130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/data/info.pb
I20260812 03:57:05.879076 11206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:05.880717 11206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:05.885905 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 11130
I20260812 03:57:05.885998 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-0/wal/instance
I20260812 03:57:05.886224 17627 external_mini_cluster.cc:1428] Setting key d6d029b013e87d96242b6e660c0f1d5a
Aug 12 03:57:05 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507025, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:57:05.892076 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:05.881522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.17.54.254:37203)
0812 03:57:05.881679 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:05.881695 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:05.881797 (+ 102us) server_negotiation.cc:407] Connection header received
0812 03:57:05.882361 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:05.882368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:05.882386 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:05.882492 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:05.883043 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:05.883884 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:05.885009 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:05.885430 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:05.885541 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:05.888402 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:05.888409 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:05.888410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:05.888417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:05.890193 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:05.890794 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:05.890799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:05.890799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:05.890925 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:05.891217 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:05.891217 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:05.891218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:05.891446 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:05.891514 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:05.891786 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 03:57:05.891904 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260812 03:57:05.892223 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:05.881455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37203 (local address 127.0.0.1:59612)
0812 03:57:05.881660 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:05.881702 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:05.882184 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:05.882523 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:05.882525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:05.882779 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:05.882955 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:05.882959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:05.884000 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:05.884001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:05.884905 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:05.884908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:05.885557 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:05.885558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:05.885615 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:05.886268 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:05.886276 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:05.888020 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:05.890323 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:05.890332 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:05.890334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:05.890649 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:05.891014 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:05.891015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:05.891015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:05.891138 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:05.891526 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:05.891534 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:05.891620 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 03:57:05.891737 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:57:05.892033 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":92,"thread_start_us":123,"threads_started":1}
I20260812 03:57:05.893112 11206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:05.893559 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:05.893419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49306)
0812 03:57:05.893476 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:05.893532 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:57:05.893729 11150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
I20260812 03:57:05.894241 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:05.894124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49308)
0812 03:57:05.894179 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:05.894214 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:57:05.894426 11206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:45497: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.54.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab.
Entry for principal kudu/127.17.54.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab.
I20260812 03:57:05.915521 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.253:45497
--webserver_interface=127.17.54.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab
--principal=kudu/127.17.54.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:37203,127.17.54.253:45497,127.17.54.252:40303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:57:05.943703 11206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } attempt: 1
I20260812 03:57:05.944298 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:05.944119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49312)
0812 03:57:05.944200 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:05.944255 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:57:05.944777 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:05.944675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49328)
0812 03:57:05.944724 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:05.944753 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:57:05.944927 11206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:45497: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
W20260812 03:57:06.000648 11218 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:57:06.000905 11218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:06.000936 11218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:06.000978 11218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:06.002116 11218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:06.002202 11218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:06.002229 11218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:06.002260 11218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:06.002283 11218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:06.003801 11218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:37203,127.17.54.253:45497,127.17.54.252:40303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.253.keytab
--principal=kudu/127.17.54.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.253:45497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.54.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:06.004087 11218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:06.004333 11218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:06.009586 11226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:06.009604 11228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:06.009639 11223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:06.009958 11218 server_base.cc:1034] running on GCE node
I20260812 03:57:06.010285 11218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:06.010687 11218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:06.011870 11218 hybrid_clock.cc:648] HybridClock initialized: now 1786507026011852 us; error 47 us; skew 500 ppm
I20260812 03:57:06.013082 11206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } attempt: 2
I20260812 03:57:06.013566 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.013400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49330)
0812 03:57:06.013471 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.013521 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 03:57:06.013998 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.013867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49338)
0812 03:57:06.013942 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.013973 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507026, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:57:06.014695 11206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.253:45497: Network error: Client connection negotiation failed: client connection to 127.17.54.253:45497: connect: Connection refused (error 111)
I20260812 03:57:06.014959 11218 init.cc:377] Logged in from keytab as kudu/127.17.54.253@KRBTEST.COM (short username kudu)
I20260812 03:57:06.015429 11218 webserver.cc:460] Webserver started at http://127.17.54.253:42037/ using document root <none> and password file <none>
I20260812 03:57:06.015677 11218 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:06.015748 11218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:06.015914 11218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:06.017035 11218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/data/instance:
uuid: "e52c7f7908c04e508613ffe673446362"
format_stamp: "Formatted at 2026-08-12 03:57:06 on dist-test-slave-tm1g"
server_key: "d2be7bf4f6159b0616cd2b3dce0cab65"
server_key_iv: "9a5539137469154f5ac5a09604d30983"
server_key_version: "encryptionkey@0"
I20260812 03:57:06.017419 11218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/wal/instance:
uuid: "e52c7f7908c04e508613ffe673446362"
format_stamp: "Formatted at 2026-08-12 03:57:06 on dist-test-slave-tm1g"
server_key: "d2be7bf4f6159b0616cd2b3dce0cab65"
server_key_iv: "9a5539137469154f5ac5a09604d30983"
server_key_version: "encryptionkey@0"
I20260812 03:57:06.018961 11218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:06.019903 11235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:06.020151 11218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:06.020272 11218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/wal
uuid: "e52c7f7908c04e508613ffe673446362"
format_stamp: "Formatted at 2026-08-12 03:57:06 on dist-test-slave-tm1g"
server_key: "d2be7bf4f6159b0616cd2b3dce0cab65"
server_key_iv: "9a5539137469154f5ac5a09604d30983"
server_key_version: "encryptionkey@0"
I20260812 03:57:06.020352 11218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:06.042227 11218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:06.044564 11218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:06.044751 11218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:06.049088 11218 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.253:45497
I20260812 03:57:06.049173 11292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.253:45497 every 8 connection(s)
I20260812 03:57:06.049587 11218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/data/info.pb
I20260812 03:57:06.050870 11293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:06.051563 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 11218
I20260812 03:57:06.051642 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-1/wal/instance
I20260812 03:57:06.051858 17627 external_mini_cluster.cc:1428] Setting key f89451dedc3fb12c3ce70117e426814f
I20260812 03:57:06.052801 11293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507026, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.54.252@KRBTEST.COM; defaulting to no policy
I20260812 03:57:06.064357 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.053649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59622 (local address 127.17.54.254:37203)
0812 03:57:06.053723 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.053726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.053999 (+ 273us) server_negotiation.cc:407] Connection header received
0812 03:57:06.054901 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.054902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.054956 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.055067 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.055724 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.056647 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.057872 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.058294 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.058412 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.060774 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.060775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.060776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.060786 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.062587 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.063222 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.063223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.063223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.063285 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.063568 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.063569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.063570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.063707 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.063775 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.064178 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.064260 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:57:06.064436 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.053613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37203 (local address 127.0.0.1:59622)
0812 03:57:06.053842 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.053901 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:06.054755 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:06.055148 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:06.055152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:06.055411 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:06.055611 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.055615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.056769 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.056770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.057749 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.057752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.058451 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.058453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.058582 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.059220 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:06.059229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:06.060664 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:06.062842 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.062851 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.062853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.063127 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.063362 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.063363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.063363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.063478 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.063777 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:06.063782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:06.064024 (+ 242us) client_negotiation.cc:769] Sending connection context
0812 03:57:06.064118 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:57:06.064253 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":144,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.54.252@KRBTEST.COM" created.
I20260812 03:57:06.065138 11293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507026, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:57:06.072773 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.065551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49348 (local address 127.17.54.253:45497)
0812 03:57:06.065725 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.065729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.065746 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:57:06.065762 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.065764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.065786 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.065906 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.066182 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.066947 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.067644 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.067980 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.068080 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.069905 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.069914 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.069914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.069921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.071533 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.071908 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.071911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.071911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.071970 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.072169 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.072169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.072170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.072320 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.072436 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.072550 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.072634 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260812 03:57:06.072777 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.065470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49348)
0812 03:57:06.065529 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.065542 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:06.065589 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:06.065909 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:06.065910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:06.066056 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:06.066104 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.066105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.067059 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.067060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.067561 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.067562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.068084 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.068085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.068152 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.068605 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:06.068610 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:06.069796 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:06.071637 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.071638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.071639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.071807 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.072039 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.072039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.072040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.072096 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.072414 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:06.072415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:06.072456 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:57:06.072521 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:57:06.072682 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":71}
I20260812 03:57:06.073604 11293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:06.074010 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.073892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41676)
0812 03:57:06.073943 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.073988 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:57:06.074225 11238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
I20260812 03:57:06.074729 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.074640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41690)
0812 03:57:06.074680 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.074706 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 03:57:06.074848 11293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:40303: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.54.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab.
Entry for principal kudu/127.17.54.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab.
I20260812 03:57:06.082134 17627 external_mini_cluster.cc:1326] Running /tmp/dist-test-task75kCRI/build/release/bin/kudu
/tmp/dist-test-task75kCRI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.252:40303
--webserver_interface=127.17.54.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab
--principal=kudu/127.17.54.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.54.212:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.54.254:37203,127.17.54.253:45497,127.17.54.252:40303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:57:06.105883 11206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } attempt: 3
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507025, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:57:06.114046 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.106394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49364)
0812 03:57:06.106450 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.106465 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:06.106535 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:06.106710 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:06.106711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:06.106884 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:06.106958 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.106960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.107942 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.107944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.108543 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.108545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.109087 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.109089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.109163 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.109731 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:06.109736 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:06.111067 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:06.112945 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.112947 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.112947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.113141 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.113358 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.113359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.113359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.113453 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.113756 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:06.113757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:06.113805 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:57:06.113876 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:57:06.113970 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 03:57:06.114105 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.106458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.17.54.253:45497)
0812 03:57:06.106525 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.106527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.106542 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:57:06.106613 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.106614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.106629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.106694 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.107036 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.107826 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.108633 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.108987 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.109063 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.111161 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.111162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.111163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.111171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.112834 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.113220 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.113221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.113222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.113285 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.113517 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.113517 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.113518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.113660 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.113732 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.113918 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.114010 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:57:06.114732 11206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:06.115105 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.114985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41696)
0812 03:57:06.115041 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.115077 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 03:57:06.115545 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.115439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41698)
0812 03:57:06.115494 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.115523 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 03:57:06.115695 11206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:40303: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
I20260812 03:57:06.124068 11293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } attempt: 1
I20260812 03:57:06.124544 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.124402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41714)
0812 03:57:06.124467 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.124512 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:57:06.124913 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.124819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41730)
0812 03:57:06.124866 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.124894 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 03:57:06.125025 11293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:40303: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
I20260812 03:57:06.146905 11206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } attempt: 1
I20260812 03:57:06.147495 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.147317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41732)
0812 03:57:06.147394 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.147452 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 03:57:06.148002 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.147870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41740)
0812 03:57:06.147921 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.147953 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:57:06.148121 11206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:40303: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
W20260812 03:57:06.169210 11300 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:57:06.169422 11300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:57:06.169440 11300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:57:06.169458 11300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:57:06.170735 11300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:57:06.170779 11300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:57:06.170794 11300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:57:06.170809 11300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:57:06.170820 11300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:57:06.172358 11300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.54.212:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.54.254:37203,127.17.54.253:45497,127.17.54.252:40303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/krb5kdc/kudu_127.17.54.252.keytab
--principal=kudu/127.17.54.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.54.252:40303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.54.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:57:06.172624 11300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:57:06.172864 11300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:57:06.178216 11306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:57:06.178222 11305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:06.178406 11300 server_base.cc:1034] running on GCE node
W20260812 03:57:06.178222 11309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:57:06.178730 11300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:57:06.179055 11300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:57:06.180253 11300 hybrid_clock.cc:648] HybridClock initialized: now 1786507026180243 us; error 51 us; skew 500 ppm
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507026, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:57:06.183030 11300 init.cc:377] Logged in from keytab as kudu/127.17.54.252@KRBTEST.COM (short username kudu)
I20260812 03:57:06.183555 11300 webserver.cc:460] Webserver started at http://127.17.54.252:42487/ using document root <none> and password file <none>
I20260812 03:57:06.183789 11300 fs_manager.cc:359] Metadata directory not provided
I20260812 03:57:06.183856 11300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:57:06.184018 11300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:57:06.185129 11300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/data/instance:
uuid: "c39c4d49169242c8921e7cbe50ab8500"
format_stamp: "Formatted at 2026-08-12 03:57:06 on dist-test-slave-tm1g"
server_key: "9ec3844be681b1e6f395ffd48c311103"
server_key_iv: "103080d4376d7b3295cab0d409c8799d"
server_key_version: "encryptionkey@0"
I20260812 03:57:06.185526 11300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/wal/instance:
uuid: "c39c4d49169242c8921e7cbe50ab8500"
format_stamp: "Formatted at 2026-08-12 03:57:06 on dist-test-slave-tm1g"
server_key: "9ec3844be681b1e6f395ffd48c311103"
server_key_iv: "103080d4376d7b3295cab0d409c8799d"
server_key_version: "encryptionkey@0"
I20260812 03:57:06.187057 11300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:57:06.187887 11316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:57:06.188074 11300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:57:06.188158 11300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/data,/tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/wal
uuid: "c39c4d49169242c8921e7cbe50ab8500"
format_stamp: "Formatted at 2026-08-12 03:57:06 on dist-test-slave-tm1g"
server_key: "9ec3844be681b1e6f395ffd48c311103"
server_key_iv: "103080d4376d7b3295cab0d409c8799d"
server_key_version: "encryptionkey@0"
I20260812 03:57:06.188211 11300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:57:06.193173 11293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } attempt: 2
I20260812 03:57:06.193667 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.193500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41754)
0812 03:57:06.193584 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.193636 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 03:57:06.194015 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.193925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41762)
0812 03:57:06.193968 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.193995 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:57:06.194119 11293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.54.252:40303: Network error: Client connection negotiation failed: client connection to 127.17.54.252:40303: connect: Connection refused (error 111)
I20260812 03:57:06.211069 11300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:57:06.213253 11300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:57:06.213461 11300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:57:06.217967 11300 rpc_server.cc:307] RPC server started. Bound to: 127.17.54.252:40303
I20260812 03:57:06.218055 11375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.54.252:40303 every 8 connection(s)
I20260812 03:57:06.218477 11300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/data/info.pb
I20260812 03:57:06.219509 11376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:57:06.221163 11376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:06.223238 11206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } attempt: 2
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507026, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:57:06.228500 17627 external_mini_cluster.cc:1388] Started /tmp/dist-test-task75kCRI/build/release/bin/kudu as pid 11300
I20260812 03:57:06.228603 17627 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task75kCRI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506741478267-17627-0/minicluster-data/master-2/wal/instance
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507025, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.254@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
I20260812 03:57:06.228824 17627 external_mini_cluster.cc:1428] Setting key b4e9ae61ccab9bccd9bfd5fea61b3b29
I20260812 03:57:06.231624 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.221934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59632 (local address 127.17.54.254:37203)
0812 03:57:06.222004 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.222006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.222241 (+ 235us) server_negotiation.cc:407] Connection header received
0812 03:57:06.222814 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.222816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.222847 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.222911 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.223708 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.224236 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.225250 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.225478 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.225577 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.228068 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.228069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.228069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.228077 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.229875 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.230513 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.230514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.230515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.230582 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.230857 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.230858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.230859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.231022 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.231103 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.231472 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.231552 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:57:06.231818 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.221855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.254:37203 (local address 127.0.0.1:59632)
0812 03:57:06.222066 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.222110 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:06.222643 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:06.223005 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:06.223009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:06.223321 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:06.223577 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.223581 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.224370 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.224371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.225160 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.225162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.225583 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.225589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.225712 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.226288 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:06.226296 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:06.227967 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:06.230001 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.230012 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.230014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.230418 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.230652 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.230653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.230653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.230770 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.231094 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:06.231101 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:06.231283 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 03:57:06.231421 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 03:57:06.231575 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260812 03:57:06.232427 11376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:06.233052 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.223640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41768)
0812 03:57:06.223701 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.223713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:06.223760 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:06.224054 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:06.224055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:06.224273 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:06.224319 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.224321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.225391 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.225392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.226497 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.226500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.227036 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.227037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.227081 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.227526 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:06.227531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:06.229067 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:06.231784 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.231786 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.231787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.232028 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.232250 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.232251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.232251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.232322 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.232732 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:06.232733 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:06.232782 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 03:57:06.232841 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:57:06.232946 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 03:57:06.233541 11379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.223715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.17.54.252:40303)
0812 03:57:06.223898 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.223902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.223919 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:57:06.223937 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.223939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.223954 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.224044 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.224413 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.225299 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.226599 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.226953 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.227027 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.229195 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.229206 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.229206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.229215 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.231313 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.232114 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.232117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.232117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.232180 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.232411 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.232412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.232413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.232624 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.233310 (+ 686us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.233361 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.233460 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260812 03:57:06.235293 11206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0: Bootstrap starting.
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
I20260812 03:57:06.236156 11206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:06.236480 11206 log.cc:824] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507026, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:57:06.238030 11206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0: No bootstrap required, opened a new log
I20260812 03:57:06.239733 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.229282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.17.54.252:40303)
0812 03:57:06.230245 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.230247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.230254 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:57:06.230276 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.230277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.230288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.230378 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.231253 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.231969 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.233016 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.233326 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.233380 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.236283 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.236285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.236285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.236304 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.238524 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.238869 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.238870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.238871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.238929 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.239143 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.239144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.239144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.239270 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.239431 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.239494 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.239580 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":84,"threads_started":1}
I20260812 03:57:06.239701 11206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } }
I20260812 03:57:06.239862 11206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5773951dc9c54dadafbbb1eb1766a6a0, State: Initialized, Role: FOLLOWER
I20260812 03:57:06.239979 11206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } }
I20260812 03:57:06.240296 11383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.240381 11206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:06.240411 11383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:57:06.242475 11396 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:06.242537 11396 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:06.245995 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.232764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49374 (local address 127.17.54.253:45497)
0812 03:57:06.232815 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.232817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.232825 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:57:06.232853 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.232854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.232864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.232936 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.233251 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.233950 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.235057 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.235304 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.235339 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.242394 (+ 7055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.242396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.242396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.242406 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.244411 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.244933 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.244934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.244935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.245013 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.245283 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.245304 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.245305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.245473 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.245527 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.245768 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.245861 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:57:06.243813 11130 master_runner.cc:186] Error getting master registration for 127.17.54.253:45497: OK, instance_id { permanent_uuid: "e52c7f7908c04e508613ffe673446362" instance_seqno: 1786507026047812 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:57:06.246284 11130 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:57:06.246387 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.232672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.253:45497 (local address 127.0.0.1:49374)
0812 03:57:06.232708 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.232716 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:06.232745 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:06.232954 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:06.232955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:06.233107 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:06.233154 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.233155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.234072 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.234073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.234943 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.234947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.235439 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.235440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.235523 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.236321 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:06.236327 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:06.242245 (+ 5918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:06.244548 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.244549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.244549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.244815 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.245105 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.245106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.245107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.245191 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.245588 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:06.245589 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:06.245644 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:57:06.246123 (+ 479us) client_negotiation.cc:240] Negotiation successful
0812 03:57:06.246228 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.254@KRBTEST.COM
I20260812 03:57:06.246842 11376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:57:06.250944 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.240476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.17.54.254:37203)
0812 03:57:06.240513 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.240515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.240572 (+ 57us) server_negotiation.cc:407] Connection header received
0812 03:57:06.240660 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.240661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.240668 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.240707 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.241391 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.242046 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.243397 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.243993 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.244083 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.247250 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.247251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.247251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.247259 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.249085 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.249522 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.249523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.249524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.249586 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.249825 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.249826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.249826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.249976 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.250139 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.250477 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.250571 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507026, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.252@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
I20260812 03:57:06.255359 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.247104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41790)
0812 03:57:06.247230 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.247238 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:06.247291 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:06.247465 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:06.247466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:06.247616 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:06.247655 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.247657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.248701 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.248703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.249576 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.249580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.250437 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.250438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.250489 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.250884 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:06.250890 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:06.252382 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:06.254023 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.254024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.254025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.254270 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.254534 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.254535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.254536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.254609 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.254976 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:06.254977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:06.255030 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:57:06.255106 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:57:06.255235 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260812 03:57:06.255421 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.247179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.17.54.252:40303)
0812 03:57:06.247211 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.247213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.247271 (+ 58us) server_negotiation.cc:407] Connection header received
0812 03:57:06.247362 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.247364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.247377 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.247456 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.247810 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.248606 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.249853 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.250242 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.250373 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.252486 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.252487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.252488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.252495 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.253931 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.254379 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.254380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.254381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.254450 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.254686 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.254687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.254687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.254893 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.254968 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.255161 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.255293 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":16}
I20260812 03:57:06.256992 11376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500: Bootstrap starting.
I20260812 03:57:06.257834 11376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500: Neither blocks nor log segments found. Creating new log.
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.54.253@KRBTEST.COM
I20260812 03:57:06.258190 11376 log.cc:824] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:06.259176 11376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500: No bootstrap required, opened a new log
I20260812 03:57:06.261289 11376 raft_consensus.cc:348] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } }
I20260812 03:57:06.261474 11376 raft_consensus.cc:729] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c39c4d49169242c8921e7cbe50ab8500, State: Initialized, Role: FOLLOWER
I20260812 03:57:06.261579 11376 consensus_queue.cc:260] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } }
I20260812 03:57:06.261965 11376 sys_catalog.cc:564] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:06.262073 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.251535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49388 (local address 127.17.54.253:45497)
0812 03:57:06.251583 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.251585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.252215 (+ 630us) server_negotiation.cc:407] Connection header received
0812 03:57:06.252267 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.252268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.252281 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.252350 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.253084 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.254071 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.255062 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.255370 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.255437 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.258750 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.258752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.258752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.258760 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.260542 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.261012 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.261013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.261013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.261076 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.261319 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.261320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.261320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.261464 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.261623 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.261822 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.261914 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:57:06.263717 11401 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.263876 11401 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:06.264349 17627 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:57:06.265383 11178 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5773951dc9c54dadafbbb1eb1766a6a0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59638
I20260812 03:57:06.265561 11178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:57:06.265620 11178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [term 0 FOLLOWER]: Advancing to term 1
W20260812 03:57:06.265997 11416 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:57:06.266120 11416 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:06.266825 11178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } }
I20260812 03:57:06.267102 11178 leader_election.cc:288] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [CANDIDATE]: Term 1 election: Requested vote from peers e52c7f7908c04e508613ffe673446362 (127.17.54.253:45497), c39c4d49169242c8921e7cbe50ab8500 (127.17.54.252:40303)
I20260812 03:57:06.268601 11347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c39c4d49169242c8921e7cbe50ab8500"
I20260812 03:57:06.266554 11300 master_runner.cc:186] Error getting master registration for 127.17.54.253:45497: OK, instance_id { permanent_uuid: "e52c7f7908c04e508613ffe673446362" instance_seqno: 1786507026047812 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:57:06.268733 11347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:06.268811 11300 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:57:06.269822 11347 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5773951dc9c54dadafbbb1eb1766a6a0 in term 1.
I20260812 03:57:06.270104 11149 leader_election.cc:302] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5773951dc9c54dadafbbb1eb1766a6a0, c39c4d49169242c8921e7cbe50ab8500; no voters:
I20260812 03:57:06.270259 11383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:57:06.270467 11383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [term 1 LEADER]: Becoming Leader. State: Replica: 5773951dc9c54dadafbbb1eb1766a6a0, State: Running, Role: LEADER
I20260812 03:57:06.270576 11383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } }
I20260812 03:57:06.271121 11422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5773951dc9c54dadafbbb1eb1766a6a0. Latest consensus state: current_term: 1 leader_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.271215 11422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:06.282236 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.270826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.17.54.252:40303)
0812 03:57:06.270936 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.270938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.272621 (+ 1683us) server_negotiation.cc:407] Connection header received
0812 03:57:06.272726 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.272727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.272739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.272772 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.274726 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.275519 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.276701 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.277041 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.277112 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.279177 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.279178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.279179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.279186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.280862 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.281378 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.281379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.281380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.281442 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.281641 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.281642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.281642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.281770 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.281844 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.282048 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.282134 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 03:57:06.282680 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.270289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.17.54.253:45497)
0812 03:57:06.270384 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.270387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.271491 (+ 1104us) server_negotiation.cc:407] Connection header received
0812 03:57:06.271608 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.271610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.271622 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.271655 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.272524 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.273270 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.274673 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.275059 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.275130 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.278758 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.278759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.278760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.278768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.280643 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.281417 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.281418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.281419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.281485 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.281817 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.281818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.281819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.281987 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.282075 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.282478 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.282587 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:57:06.285892 11293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } attempt: 3
I20260812 03:57:06.286206 11427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:57:06.286609 11427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:57:06.289146 11347 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [term 1 FOLLOWER]: Refusing update from remote peer 5773951dc9c54dadafbbb1eb1766a6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:06.289518 11422 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:06.289899 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.271152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59640 (local address 127.17.54.254:37203)
0812 03:57:06.271200 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.271202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.277701 (+ 6499us) server_negotiation.cc:407] Connection header received
0812 03:57:06.277825 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.277826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.277840 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.277876 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.278624 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.280447 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.281598 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.281928 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.281973 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.284046 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.284047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.284047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.284055 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.285792 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.287070 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.287072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.287072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.287139 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.287392 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.287393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.287393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.287546 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.287650 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.289671 (+ 2021us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.289769 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:57:06.290937 11401 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5773951dc9c54dadafbbb1eb1766a6a0. Latest consensus state: current_term: 1 leader_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.291036 11401 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 03:57:06 dist-test-slave-tm1g krb5kdc[11115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786507026, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.54.253@KRBTEST.COM for kudu/127.17.54.252@KRBTEST.COM
I20260812 03:57:06.292213 11422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.292312 11422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:06.292723 11401 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.292827 11401 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:06.292963 11428 mvcc.cc:205] Tried to move back new op lower bound from 7317532779676467200 to 7317532779605213184. Current Snapshot: MvccSnapshot[applied={T|T < 7317532779676467200}]
I20260812 03:57:06.293799 11427 catalog_manager.cc:1347] Generated new cluster ID: bca9c0f026324625b7e773f3200411f1
I20260812 03:57:06.293886 11427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:57:06.295097 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.286240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.17.54.252:40303)
0812 03:57:06.286287 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.286289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.286322 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:57:06.286402 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.286402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.286412 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.286472 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.286779 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.287526 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.288706 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.289050 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.289127 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.291574 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:57:06.291576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:57:06.291576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:57:06.291584 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:57:06.293707 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.294250 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.294251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.294251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.294350 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:57:06.294579 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:57:06.294579 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:57:06.294580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:57:06.294722 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:57:06.294803 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:06.294926 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 03:57:06.295005 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:57:06.295112 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:06.286183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.54.252:40303 (local address 127.0.0.1:41810)
0812 03:57:06.286240 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 03:57:06.286252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:57:06.286293 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:57:06.286480 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:57:06.286481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:57:06.286646 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:57:06.286697 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.286699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.287692 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.287693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.288595 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:57:06.288598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:57:06.289132 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:57:06.289133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:57:06.289210 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:06.289778 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:57:06.289783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:57:06.291469 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:57:06.293881 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.293882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.293882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.294152 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.294419 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:57:06.294419 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:57:06.294420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:57:06.294496 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:57:06.294783 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:57:06.294784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:57:06.294832 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:57:06.294900 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:57:06.295011 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:57:06.296965 11293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362: Bootstrap starting.
I20260812 03:57:06.297715 11293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362: Neither blocks nor log segments found. Creating new log.
I20260812 03:57:06.298012 11293 log.cc:824] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362: Log is configured to *not* fsync() on all Append() calls
I20260812 03:57:06.298933 11293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362: No bootstrap required, opened a new log
I20260812 03:57:06.300964 11293 raft_consensus.cc:348] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } }
I20260812 03:57:06.301112 11293 raft_consensus.cc:729] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52c7f7908c04e508613ffe673446362, State: Initialized, Role: FOLLOWER
I20260812 03:57:06.301192 11293 consensus_queue.cc:260] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } }
I20260812 03:57:06.301515 11293 sys_catalog.cc:564] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:57:06.301662 11262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:57:06.302663 11262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [term 1 FOLLOWER]: Refusing update from remote peer 5773951dc9c54dadafbbb1eb1766a6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:57:06.302827 11430 mvcc.cc:205] Tried to move back new op lower bound from 7317532779676467200 to 7317532779605213184. Current Snapshot: MvccSnapshot[applied={T|T < 7317532779676467200}]
I20260812 03:57:06.302987 11383 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:57:06.303638 11263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e52c7f7908c04e508613ffe673446362"
I20260812 03:57:06.304906 11263 raft_consensus.cc:2396] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5773951dc9c54dadafbbb1eb1766a6a0 for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 03:57:06.305583 11450 mvcc.cc:205] Tried to move back new op lower bound from 7317532779676467200 to 7317532779605213184. Current Snapshot: MvccSnapshot[applied={T|T < 7317532779676467200}]
I20260812 03:57:06.305711 11448 sys_catalog.cc:455] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.305816 11448 sys_catalog.cc:458] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:06.305874 11448 sys_catalog.cc:455] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5773951dc9c54dadafbbb1eb1766a6a0. Latest consensus state: current_term: 1 leader_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.305925 11448 sys_catalog.cc:458] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:06.305963 11448 sys_catalog.cc:455] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.306023 11448 sys_catalog.cc:458] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:57:06.307188 11447 catalog_manager.cc:1259] Loaded cluster ID: bca9c0f026324625b7e773f3200411f1
I20260812 03:57:06.307257 11447 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362: loading cluster ID for follower catalog manager: success
W20260812 03:57:06.307363 11447 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260812 03:57:06.307415 11447 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:57:06.320441 11383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5773951dc9c54dadafbbb1eb1766a6a0" member_type: VOTER last_known_addr { host: "127.17.54.254" port: 37203 } } peers { permanent_uuid: "e52c7f7908c04e508613ffe673446362" member_type: VOTER last_known_addr { host: "127.17.54.253" port: 45497 } } peers { permanent_uuid: "c39c4d49169242c8921e7cbe50ab8500" member_type: VOTER last_known_addr { host: "127.17.54.252" port: 40303 } } }
I20260812 03:57:06.320575 11383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:57:06.324626 11427 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:57:06.325543 11427 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:57:06.333611 11427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5773951dc9c54dadafbbb1eb1766a6a0: Generated new TSK 0
I20260812 03:57:06.333926 11427 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:57:06.370209 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.367987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.17.54.254:37203)
0812 03:57:06.368041 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.368043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.369036 (+ 993us) server_negotiation.cc:407] Connection header received
0812 03:57:06.369304 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.369305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.369318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.369352 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.370126 (+ 774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59652: BlockingRecv error: recv got EOF from 127.0.0.1:59652 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 03:57:06.371068 11381 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.368248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.17.54.252:40303)
0812 03:57:06.368309 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.368311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.369136 (+ 825us) server_negotiation.cc:407] Connection header received
0812 03:57:06.369547 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.369548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.369561 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.369593 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.370997 (+ 1404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41824: BlockingRecv error: recv got EOF from 127.0.0.1:41824 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 03:57:06.371960 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.368465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49402 (local address 127.17.54.253:45497)
0812 03:57:06.368508 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.368510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.369489 (+ 979us) server_negotiation.cc:407] Connection header received
0812 03:57:06.371448 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.371450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.371463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.371495 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.371895 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49402: BlockingRecv error: recv got EOF from 127.0.0.1:49402 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 03:57:06.374955 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.374038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49412 (local address 127.17.54.253:45497)
0812 03:57:06.374080 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.374082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.374342 (+ 260us) server_negotiation.cc:407] Connection header received
0812 03:57:06.374473 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.374474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.374495 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.374526 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.374903 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49412: BlockingRecv error: recv got EOF from 127.0.0.1:49412 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 03:57:06.378393 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.377439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49416 (local address 127.17.54.253:45497)
0812 03:57:06.377479 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.377481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.377756 (+ 275us) server_negotiation.cc:407] Connection header received
0812 03:57:06.377874 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.377875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.377888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.377919 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.378275 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49416: BlockingRecv error: recv got EOF from 127.0.0.1:49416 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 03:57:06.382783 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.381724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49420 (local address 127.17.54.253:45497)
0812 03:57:06.381765 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.381767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.382085 (+ 318us) server_negotiation.cc:407] Connection header received
0812 03:57:06.382193 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.382194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.382206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.382236 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.382721 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49420: BlockingRecv error: recv got EOF from 127.0.0.1:49420 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 03:57:06.389195 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.388206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49432 (local address 127.17.54.253:45497)
0812 03:57:06.388257 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.388259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.388567 (+ 308us) server_negotiation.cc:407] Connection header received
0812 03:57:06.388675 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.388676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.388688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.388719 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.389138 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49432: BlockingRecv error: recv got EOF from 127.0.0.1:49432 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 03:57:06.399880 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.398768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49448 (local address 127.17.54.253:45497)
0812 03:57:06.398820 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.398823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.399162 (+ 339us) server_negotiation.cc:407] Connection header received
0812 03:57:06.399278 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.399280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.399295 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.399327 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.399797 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49448: BlockingRecv error: recv got EOF from 127.0.0.1:49448 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 03:57:06.418563 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.417466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.17.54.253:45497)
0812 03:57:06.417515 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.417519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.417854 (+ 335us) server_negotiation.cc:407] Connection header received
0812 03:57:06.417972 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.417974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.417987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.418020 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.418500 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49454: BlockingRecv error: recv got EOF from 127.0.0.1:49454 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 03:57:06.452983 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.452178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49460 (local address 127.17.54.253:45497)
0812 03:57:06.452239 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.452241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.452393 (+ 152us) server_negotiation.cc:407] Connection header received
0812 03:57:06.452467 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.452468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.452481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.452557 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.452923 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49460: BlockingRecv error: recv got EOF from 127.0.0.1:49460 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 03:57:06.519172 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.518348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49466 (local address 127.17.54.253:45497)
0812 03:57:06.518559 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.518562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.518579 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:57:06.518609 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.518610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.518623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.518700 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.519120 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49466: BlockingRecv error: recv got EOF from 127.0.0.1:49466 (error 108)
Metrics: {"server-negotiator.queue_time_us":178}
W20260812 03:57:06.649652 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.648694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.17.54.253:45497)
0812 03:57:06.648790 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.648792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.648926 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:57:06.649041 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.649043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.649062 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.649126 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.649576 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49470: BlockingRecv error: recv got EOF from 127.0.0.1:49470 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 03:57:06.907749 11297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:57:06.907024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49486 (local address 127.17.54.253:45497)
0812 03:57:06.907097 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:06.907099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:06.907109 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:57:06.907181 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:06.907182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:06.907196 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:06.907269 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:57:06.907627 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49486: BlockingRecv error: recv got EOF from 127.0.0.1:49486 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:57:07.266687 11416 catalog_manager.cc:1259] Loaded cluster ID: bca9c0f026324625b7e773f3200411f1
I20260812 03:57:07.266774 11416 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500: loading cluster ID for follower catalog manager: success
I20260812 03:57:07.267591 11416 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500: acquiring CA information for follower catalog manager: success
I20260812 03:57:07.267855 11416 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c39c4d49169242c8921e7cbe50ab8500: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:57:07.308601 11447 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362: acquiring CA information for follower catalog manager: success
I20260812 03:57:07.309008 11447 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e52c7f7908c04e508613ffe673446362: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:57:07.423928 11532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:07.421079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49500 (local address 127.17.54.253:45497)
0812 03:57:07.421228 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:07.421230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:07.421259 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:57:07.421287 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:07.421288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:07.421344 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:07.421418 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:07.421625 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.422148 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.423027 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.423232 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.423289 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:07.423580 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:07.423704 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:07.423777 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:07.423832 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:57:07.423858 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260812 03:57:07.428759 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:07.426079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.17.54.252:40303)
0812 03:57:07.426199 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:07.426201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:07.426212 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:57:07.426322 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:07.426324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:07.426388 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:07.426477 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:07.426584 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.427076 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.427882 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.428110 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.428206 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:07.428395 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:07.428511 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:07.428607 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:07.428633 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 03:57:07.428657 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260812 03:57:07.433357 11532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:07.430649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49506 (local address 127.17.54.253:45497)
0812 03:57:07.430699 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:07.430701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:07.430788 (+ 87us) server_negotiation.cc:407] Connection header received
0812 03:57:07.430893 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:07.430895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:07.430909 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:07.430965 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:07.431108 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.431874 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.432666 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.432884 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.432969 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:07.433085 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:07.433118 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:07.433168 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:07.433244 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:57:07.433268 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:57:07.433851 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:07.430640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.17.54.252:40303)
0812 03:57:07.430684 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:07.430686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:07.430910 (+ 224us) server_negotiation.cc:407] Connection header received
0812 03:57:07.431008 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:07.431008 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:07.431019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:07.431048 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:07.431239 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.431704 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.432784 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.433093 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.433129 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:07.433355 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:07.433387 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:07.433466 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:07.433744 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 03:57:07.433769 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:57:07.434109 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:57:07.430538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.17.54.254:37203)
0812 03:57:07.430712 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 03:57:07.430714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:57:07.430770 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:57:07.430917 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:57:07.430919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:57:07.430977 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:57:07.431072 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:57:07.431197 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.431956 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.432938 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:57:07.433396 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:57:07.433460 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:57:07.433726 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:57:07.433844 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:57:07.433928 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:57:07.433983 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:57:07.434013 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260812 03:57:07.436133 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 11130
I20260812 03:57:07.445097 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 11218
I20260812 03:57:07.453377 17627 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task75kCRI/build/release/bin/kudu with pid 11300
2026-08-12T03:57:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2970 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6018 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (285973 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST